Original research••

Past, present and future of living systematic review: a bibliometrics analysis

•,,,,,,,,,,,,.

...

Abstract

Introduction In recent years, the concept of living systematic review (LSR) has attracted the attention of many scholars and institutions. A growing number of studies have been conducted based on LSR methodology, but their focus direction is unclear. The objective of this study was to provide a comprehensive review of existing LSR-related studies and to analyse their whole picture and future trends with bibliometrics.

Methods A comprehensive search strategy was used to construct a representative dataset of LSRs up to October 2021. GraphPad V.8.2.1 and Mindmaster Pro presented the basic information of the included studies and the timeline of LSR development, respectively. The author and country cooperation network, hotspot distribution clustering, historical citation network and future development trend prediction related to LSR were visualised by VOSviewer V.1.6.16 and R-Studio V.1.4.

Results A total of 213 studies were eventually included. The concept of LSR was first proposed in 2014, and the number of studies has proliferated since 2020. There was a closer collaboration between author teams and more frequent LSR research development and collaboration in Europe, North America and Australia. Numerous LSR studies have been published in high-impact journals. COVID-19 is the predominant disease of concern at this stage, and the rehabilitation of its patients and virological studies are possible directions of research in LSR for a long time to come. A review of existing studies found that more than half of the LSR series had not yet been updated and that the method needed to be more standardised in practice.

Conclusion Although LSR has a relatively short history, it has received much attention and currently has a high overall acceptance. The LSR methodology was further practised in COVID-19, and we look forward to seeing it applied in more areas.

What is already known on this topic?

Living systematic review (LSR) is continuously updated to ensure the timeliness and accuracy of evidence-based practice findings. Although seven years have passed since it was introduced, the current status of its application is unclear.
No overview studies have been conducted to analyse the whole picture of LSR or to predict its future trends.

What this study adds?

LSR has continued to receive attention since its introduction in 2014 and has proliferated since 2020, with studies applying the method generally gaining decent recognition and acceptance.
There was a wide disparity in the level of LSR conducted in different countries. In Europe, North America and Australia, more LSR research was conducted, and there was a close exchange and cooperation with the world.
COVID-19 is the main focus of LSR at this stage, and the rehabilitation of its patients and virological studies may be a future area of interest.
More than half of the studies based on LSR methodology have not yet completed at least one update, and those that have done so have varied in the form of their updates.

How this study might affect research, practice or policy?

Different institutions and countries can promote the application of LSR and strengthen the practice and cooperation.
LSR has supported the rapid evidence-based integration of COVID-19 and may be extended to more areas in the future.
Perhaps sharing criteria between the LSR series of studies could lead to greater standardisation in their production.

Introduction

Health-based clinical research is a crucial driver of human health and well-being. However, the possible differences and contradictory research results often confuse clinical caregivers, who invest in various kinds of research and fail to get their due return. Over the past 30 years, the field of evidence-based medicine (EBM) has developed a series of evidence synthesis approaches to base all health decisions on the best available evidence.1 Systematic review (SR) and meta-analysis (MA) have bridged the gap between original research and clinical practice, narrowing the gap between evidence and practice. However, SR and MA have limitations, such as the time lag in incorporating the latest clinical evidence, which reduces the accuracy and usefulness, and some subjects are not continuously maintained and followed up.2 In general, SRs are not updated or are only updated irregularly.3 The gap between updates may lead to the uncertainty of evidence recommendation.4 If the research team does not update with the new one, it will be difficult to form a new team, and a lot of completed work will also start from the beginning, resulting in the loss of memory and waste of resources.5

The concept of living systematic review (LSR) was born in 2014. It refers to a kind of SR constantly updated and incorporates relevant new evidence to ensure the timeliness and accuracy of evidence-based practice conclusions.5 6 Unlike Rapid Review, whose methodology may be modified to support faster production,7 the LSR still follows the SR’s standard methodology.5 Currently, many studies based on LSR have been published and widely echoed by the community. The Cochrane Collaboration has carried out a comprehensive and systematic interpretation of the LSR methodology.5 8–10 LSR does not exist in a vacuum either. The development of LSR has driven the emergence and rise of living guidelines,8 which is the optimisation of the traditional guidance development process to update individual recommendations as new evidence becomes available. However, there seems to be no consensus on how to update the rapidly evolving series of studies conducted based on LSR in a standard way and how to end this resource-intensive activity on time.

In recent years, the need for rapid, evidence-based decision-making has been more urgent than ever in the challenging context of the global spread of COVID-19, which has in part driven the practical application of LSR. In order to clarify the future role of this novel research method in evidence-based decision-making and medical practice, it is necessary to understand better the whole picture and emerging trends of LSR-related research. Although some experts have put forward some prospective suggestions for LSR, no specific direction has been provided for its application, and there are no evidence-based evaluations based on bibliometric methods. We have therefore conducted a comprehensive bibliometric analysis of LSR-related research, reviewing the origins and development of LSR, presenting the ongoing research hotspots and status quo through visualisation, and reflecting on the existing problems and future development directions.

Methods

Search strategy

Bibliometric data was collected using PubMed, the Cochrane Library, Embase and Web of Science database, supplemented with relevant citation information for included studies using the Scopus database as of October 2021. We used subject-related search (including title, abstract and keywords) to maximise the positioning of content related to our research topic. The primary search terms include living systematic review, living meta-analysis, living network meta-analysis, etc. Specific search strategies were available in online supplemental table 1. References to relevant studies retrieved were supplemented to extend the search.

Study scope

Since the concept of LSR was proposed, we have conducted a longitudinal analysis of its related research areas. The constraint conditions for selecting relevant research were set thus: (1) the research topic must be related to LSR; (2) the content of the paper can involve only conceptual or methodological research of LSR; (3) data synthesis research, review reports, etc based on LSR methodology; (4) no restrictions on the type of published language and research.

Criteria for studies based on LSR methodology: (1) the research proposal or methodology should mention that the research is an LSR; (2) the research needs to describe its update plan and frequency briefly; (3) the significance of living updates should be mentioned in the study.

Cases were excluded if any of the following occurred: (1) the SRs update did not focus on healthcare interventions, including disease progression, diagnosis, treatment and prognostic care; (2) some serial renewal studies changed their plans to LSR mid-stream, so researches before the change were also not be considered.

Study selection

The retrieved literature was imported into the web-based SR software, ‘Rayyan’ for screening analysis.11 All records were screened back-to-back by two independent investigators (QZ and JX) to determine whether they met our inclusion criteria. Any disputes were solved by a discussion with a third reviewer (JT). The literature screening process went through the following steps: first, duplicate literature was excluded, but the updated studies that maintained the same title, author information and year were screened carefully. Then, the remaining literature was screened by title and abstract combined with inclusion criteria. Uncertain studies were evaluated through the full text and discussion. The specific filtering details can be found in online supplemental figure 1A.

Data analysis

The annual number of articles, type of research and the number of articles published in each journal were collated using Microsoft Excel 2019 (www.microsoft.com) and visualised with GraphPad Prism V.8.2.1. We also conducted a detailed review of the included LSR-related studies, flagging significant events in history and creating a timeline of the development of the research topic through Mindmaster Pro. Existing studies based on the LSR methodology were also categorised through the latest version of the International Classification of Diseases (ICD-11), and their actual updates were counted along with the form of the updates.

VOSviewer (Leiden University, Leiden, Netherlands) is a scientific knowledge mapping software tool that uses network data (mainly document knowledge units) to construct and visually analyse relationships, map scientific knowledge and display relationships such as structure, evolution and cooperation.12 The screened LSR-related studies were imported into VOSviewer software, and their authors and hot keywords were collected and sorted out. After the synonymisation and threshold setting was implemented, the cooperative relationship between high-frequency keywords and high-yield authors was mapped, and a network cluster graph was formed. The cluster graph was composed of network nodes and the connecting lines between them. The nodes represented the analysed elements such as authors and keywords, the size of which represented the frequency of occurrence of the node, the lines between nodes represented node co-occurrence or related cooperative relationships and the colour represented different clustering relationships.13 14

Bibliometrix, a scientific bibliometric software based on R language, can statistically analyse the index of relevant scientific literature and conduct research and visual processing in aspects of cocitation, coupling and co-word analysis by constructing data matrices.15 After importing the contents of LSR-related research into the Bibliometrix program in the format of BibTex, the interactive menu could present the cooperation between countries, historical citation relationship, annual hot trends and hot strategic matrix of LSR-related research.

Patient and public involvement

Patients and public were not directly involved in this review; we used publicly available data for the analysis.

Results

Publication outputs and a brief history of LSR development

A total of 213 studies were published in four languages, most of which were published in English (97.7%), three in Spanish (1.4%), one in Italian (0.5%) and one in Chinese (0.5%). The primary forms of research papers were Article and Review, which accounted for 79.8% of the total papers. Other forms of publication included conference summaries, letters, notes, etc (online supplemental figure 1B). The concept of LSR was first proposed in 2014. From then on to 2019, only a few relevant studies were published, showing a slow growth rate. The explosive growth of LSR-related studies in 2020 and 2021 far exceeded the total number of previous publications combined (online supplemental figure 1C).

We combed through the timeline of the occurrence and development of LSR (figure 1). In February 2014, the concept of LSR was first introduced in PLoS Med by Elliott with his team to bridge the gap between evidence and practice.16 Subsequently, the LSR methodology was first used in the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI),17 and it was continuously covered in subsequent updates. The project was dedicated to the integrated treatment and management of patients with TBI. The CENTER-TBI programme has completed the release of five TBI series LSRs, all of which were distributed in the J Neurotrauma and the CENTER-TBI online website (https://www.center-tbi.eu).18–22 Following this, the framework of living network meta-analysis (LNMA) was proposed in 2016, providing complete and up-to-date evidence for comparing all interventions for a particular disease. In February 2016, the Cochrane Living Evidence Network was launched, furthering the LSR research process. An online website (https://community.cochrane.org/review-production/production-resources/living-systematic-reviews) was established to introduce LSR-related concepts, display and disseminate the completed research on living data synthesis, and provide learning resources such as LSR webinars. In the following years, the Cochrane Collaboration also published several LSRs and LNMA-related studies, leading the way in the field. The Cochrane Library was also the platform for retrieving the most LSR-related studies. In 2017, the Cochrane Dynamic Evidence Collaboration published four consecutive articles on LSR methodology in J Clin Epidemiol, which systematically discussed LSR from four aspects: introduction, computer-aided automation, statistical methods and problems, and living guidelines transformation.5 8–10 This laid a solid foundation for the rapid development of periodic updates of living evidence. Less than two years after the LNMA framework was proposed, the first relevant study was published, which discussed the effectiveness comparison of various measures in the second-line treatment of non-small cell lung cancer.23 In August 2019, the WHO introduced living guidelines to update recommendations on maternal and perinatal health,24 ensuring that the guidelines were up-to-date with the latest evidence, shortening the guideline development cycle.

Request permission

Figure 1

A timeline for the development of living systematic review. LNMA, living network meta-analysis; LSR, living systematic review.

The first COVID-19-related LSR study was published in April 2020 by Wynants et al, describing the validity of diagnostic and predictive models for COVID-19.25 In September of the same year, WHO issued the first relevant living guidelines to explore drug treatment and prevention of COVID-19.26 During the same period, many COVID-19-related studies based on LSR methodology continued to emerge.27–29

Author, institution and country distribution and collaboration

The author publication volume and author cluster distribution reveal the influential research groups and potential collaborative relationships. A total of 841 authors were included in 213 articles. The top 10 authors, their institutions, and the country’s distribution are shown in table 1. As a newly emerging research field, researchers were relatively concentrated, with the top 10 authors accounting for nearly half of the total publications (45.5%). Skoetz N (University of Cologne) and Rada G (Universidad Católica de Chile) have taken the lead in LSR-related research, followed by Synnot A (Monash University), Piechotta V (University of Cologne) and Ceravolo M G (University of Ancona). Half of the top 10 authors were from Italy, three were from Germany and the other two were from Chile and Australia. VOSviewer was used to conduct cluster cooperation analysis for authors of more than two articles, and 119 authors were divided into 10 categories (figure 2). Independent cooperative groups often came from the same unit, and the author’s collaboration groups were closely related to each other, while the collaboration between different collaborative groups often relied on a single scholar. Highly published scholars such as Eliott J, Synnot A, Wells G A, Chalmani A, Rada G, Ravaud P, Wang J, etc, served as core members bridging the entire author collaborative network. The current status of global LSR research progress and collaboration is shown in figure 3. The darker the country block, the more relevant studies have been carried out, and the links between countries represented partnerships. It could be found that many European countries (UK, Spain, Netherlands, Italy, Germany, etc), the USA, Canada, China, Australia, South Korea, Chile and other countries have carried out a relatively large amount of research and have collaborated more closely. However, research in many African countries, Russia, Mexico and Venezuela was relatively scarce.

Request permission

Figure 2

Author collaboration network and cluster distribution.

Request permission

Figure 3

Current situation of national research publications and cooperation.

Table 1

The distribution of the top 10 authors and their institutions and countries

Periodical distribution

A total of 213 articles were published in 69 different journals, 53 (76.8%) of which were included in SCI journals. Table 2 shows the top 10 LSR-related journals, which contributed 130 (61%) papers. Cochrane Db Syst Rev (43, 20.2%) ranked first, followed by Ann Intern Med (16, 7.5%), BMJ-Brit Med J (15, 7.0%), J Clin Epidemiol (12, 5.6%) and Eur J Phys Rehab Med (9, 4.2%). Most of the top 10 journals were from the USA (3) and the UK (4), with the rest coming from Japan, Italy and Chile. More than half of high-volume journals belong to medicine, general and internal, with six of the 10 journals having an impact factor (IF) of more than 5. The 53 SCI-indexed journals were presented as a bubble chart in online supplemental figure 2, where the size of the bubble represented the number of papers and the horizontal coordinate represented the IF. Journals with a relatively high number of LSR-related studies (≥5) were marked in red, indicating the type of journals with a high level of focus in the field.

Table 2

The top 10 journals in living systematic review-related field

Research hotspot and co-occurrence keyword clustering network analysis

We generated a hot words cloud through word frequency sorting by VOSviewer and the hot spot analysis function of the R-biblimetrix package, which can reveal the hottest content in the field (online supplemental figure 3). COVID-19 and SARS-CoV-2 have become the hottest topics in the LSR series of studies. In addition, various evidence production integration and recommendation methods such as randomised controlled trials, SR and MA, network MA and living guidelines have also received particular attention.

Of the 1,163 keywords, 104 have a frequency of three or higher. After analysis, VOSviewer categorised them into five cluster directions and formed a network connection, revealing the main areas and development direction (figure 4). The different clusters were distinguished by distinct colours, each representing the direction of a category of research hotspots. Cluster 1 is the largest group, which included 35 keywords, mainly related to drug therapy, LSR, treatment outcome, adverse effects, epidemiology, diagnosis, and quality of life, focusing on the role of LSR in disease prevalence, diagnosis and treatment, and prognosis. Cluster 2 primarily included nine keywords such as infant, child, feeding behaviour, and psychological conditioning and focuses on children’s growth and development, feeding, and psychological development. Cluster 3 focused on the methodological aspects of LSR and mentioned network MA, decision making, practice guidelines, automation, clinical protocol and evidence-based practice. Cluster 4 consisted of 32 keywords, including MA, male, female, controlled trials, outcome evaluation, drug safety, drug combination, etc, concentrating on the efficacy analysis of different genders in drug intervention in controlled trials. The last cluster focused on global trends and treatment management for COVID-19, covering SARS-CoV-2, responses, hydroxychloroquine and angiotensin receptor antagonists, etc.

Request permission

Figure 4

Clustering distribution of keywords co-occurrence hot spot network. LSR, living systematic review.

Analysis of being cited status of LSR-related studies

Table 3 shows the top 10 most frequently cited LSR-related studies. ‘Prediction models for diagnosis and prognosis of COVID-19: systematic review and critical appraisal’ published by Wynants et al in BMJ in 2020 received the most frequent citation. This research25 reported on diagnostic and prognostic models of COVID-19 using LSR, which could greatly aid clinical decision-making and COVID-19 control. Seven of the top 10 studies were related to COVID-19, and the other highly cited studies5 6 10 introduced LSR methodology. A breakdown of topical LSR-related research by year, with links to their internal citation networks, further illustrated the development process of LSR (online supplemental figure 4). The concept of LSR was proposed in 2014,6 and its systematic methodology5 8–10 was elaborated in 2017, which laid a firm foundation for the subsequent research on LSR. Many COVID-19-related studies were published in 2020 and beyond, and more citations may follow.

Table 3

Top 10 cited living systematic review-related studies

Disease distribution based on ICD-11

We discussed the specific contents of all the studies related to LSR and analysed the disease types involved based on ICD-11, presenting them in a rose plot (online supplemental figure 5). It was clear that certain infectious or parasitic diseases dominated the lion’s share of the LSR-related postings (57.7%), most of which were related to COVID-19. Second, 10 articles were related to various tumours and cancers, such as renal cell carcinoma, non-small cell carcinoma, prostate cancer, etc. Research on other diseases was scattered and not widespread, and there was room for further advancement. In addition, methodology-related articles also occupied a relatively large part (16.0%).

Update status of LSR

A collation of all LSR series articles showed that less than half of the studies were updated at least once in the study cohort (38.0%). Studies with a total publication size greater than or equal to four and updating with a specific frequency were shown in table 4, including 10 studies, half of which were published in Cochrane Database Syst Rev. The vast majority of these series have been registered for research protocols, mainly on PROSPERO or Cochrane Library. Some studies30–32 were not considered LSR at the initial registration or preliminary study but used the LSR methodology in subsequent study updates.33–35 The most frequently updated was published in Ann Intern Med, with the title ‘Risks and Impact of Angiotensin-Converting Enzyme Inhibitors or Angiotensin-Receptor Blockers on SARS-CoV-2 Infection in Adults: A Living Systematic Review’, updated in nine editions.27 36–43 The series was registered with PROSPERO, the first article was published on 4 August 2020, and the last edition was published on 27 April 2021, during which time it has been updated roughly monthly. In addition, other articles have been updated with a specific frequency, most of which were related to COVID-19, while others covered child nutrition, e-cigarette use and psoriasis. It was worth noting that few articles were updated with multiple editions and the frequency of updates was generally erratic for most series.

Table 4

Update status of continuous living systematic review studies

Trends in LSR research and themes distribution

The relationship between the changes in LSR-related research hotspots over time was shown in online supplemental figure 6, which could reflect the trends topic of LSR research. The span of the horizontal line indicated the period of word outbreaks, and the size of the shadow sphere referred to term frequency. COVID-19 and its virology-related research occupied the top position of LSR-related research. Due to the current high global epidemic, they may continue to become a hot research topic. Research on living evidence integration of neurological function prognosis of TBI has been ongoing for many years and will continue to be focused on in the future. As research into the aetiology of COVID-19 deepened and patient mortality continued to decline, the patient prognosis for rehabilitation was becoming a hot topic of research,44–46 which could also be found in the hot word cloud (online supplemental figure 3). The concept of living guidance has also received sustained attention since it was first proposed in 2017. With the promulgation of COVID-19 guidelines and the continuous improvement of more living evidence integration systems, the LSR methodology will be further improved, and more living guidelines will come into our vision to provide efficient and accurate evidence recommendations for clinical practice. Studies on cannabis mainly focus on epilepsy in children, chronic analgesia and its safety,47 but have lost momentum in the last two years.

Figure 5 shows the distribution and future trend of current research topics related to LSR in the form of strategic coordinates. The first to fourth quadrants represented motor themes, niche themes, emerging or declining themes, and basic themes. Essential topics such as TBI, living guidelines, virology research of COVID-19, patient prognosis and rehabilitation have significantly developed. EBM, GRADE (the Grades of Recommendations, Assessment, Development and Evaluation), automation, crowdsourcing and machine learning were the cornerstone of LSR research. While promoting the development of LSR-related research, they also need to be further explored. In the early stage, Elliot et al explored the effectiveness and safety of cannabis in the treatment of paediatric epilepsy with the method of LSR,48 which also provided guidance and reference for the subsequent series of LSR studies.

Request permission

Figure 5

Strategic distribution of living systematic review related research concerns.

Discussion

LSR, as a new method of SR updating, aims to break down the trade-off between rigour and currency of current methods and provide a reliable and up-to-date synthesis of evidence.49 This study reviewed 213 existing LSR-related studies and found that studies conducted based on LSR methodology have overgrown since 2020, and LSR was gradually gaining more attention and recognition by more researchers. The development level of LSR varied greatly among different countries, with Europe, North America and Australia leading the way. COVID-19-related research will continue to be the main focus of LSR for a long time. More than half of the LSR-based series has not yet been updated, and more practice details need to be standardised.

Seven years have passed since Elliott et al first proposed the LSR concept.6 The previous studies on LSR were tepid, but the outbreak of COVID-19 also triggered many related studies. However, in general, research on LSR was still relatively scarce, and the field was still in its early stages of development, with much room for growth. Throughout the development of LSR, we could find that the integration of living evidence has been expanded and applied in a short period, including the release of the LSR systematic methodology, the proposal of the LNMA framework and the development of living guidelines. After a certain period of exploration, the methodology associated with LSR has been refined through experimentation. The rapid global epidemic of COVID-19 has prompted numerous clinical trials, which raised high demands for the rapid and efficient integration of clinical evidence,50 51 essentially promoting the development of LSR-related studies and, to some extent, demonstrating the potential for LSR development.

The vast majority of published studies were in English (97.7%). As a relatively new concept, LSR has not been widely concerned by researchers. Some institutions and scholars from Italy and Germany had certain advantages in this field and dominated the research component. There was a relatively close collaboration between them, but collaboration between different groups was relatively sparse. We look forward to further collaboration to promote the development of LSR-related research. LSR research has been conducted worldwide, but some differences and imbalances remain. LSR has been studied more frequently in Europe, North America and Australia, and a great deal of cooperation was carried out with other countries. As more and more scholars worldwide gradually realise the feasibility and significance of LSR, it is bound to progress and develop even more soon. Although the concept of LSR was relatively new, its related research was still widely accepted. Most journals that published LSR research were included in SCI journals. Many flagship journals such as BMJ, Ann Intern Med, J Clin Oncol, Eur Urol, etc, have shown high acceptance.

There were few existing studies on LSR, with a relative concentration of research hotspots. The main focus was methodology development and exploration, child growth and nutrition, disease epidemiology, diagnosis treatment and prognosis, and the prevalence, treatment and prognosis of COVID-19. In the short time since the concept of LSR was introduced, scholars have been exploring which types of diseases could be substantively helped by the LSR methodology. Since the outbreak of COVID-19 in late 2019, there has been an explosion in the use of the LSR methodology. LSR could provide rapid and compelling evidence for public health outbreaks, contributing partly to disease treatment and infection control. In the global COVID-19 pandemic, the rapid integration and updating of evidence provided by LSR have contributed significantly to the pandemic’s prevention, treatment, control and prognosis, as evidenced by the high citation rate of the LSR series of studies. As a result, the LSR methodology has received increasing attention and will be applied in more innovative ways in the future. On the other hand, the feasibility and necessity of the LSR methodology have been proven by various empirical studies, which further contributed to the refinement and rigour of the LSR methodology and provided help and reference for future research and application in more areas.

According to ICD-11, current hotspots were mainly focused on certain infectious or parasitic diseases. Many previous studies and topics based on LSR methodology were updated irregularly but have been overshadowed by the strong impact of COVID-19. Methodological exploration of LSR also occupied a large area of existing research, with scholars committed to the ongoing promotion of its application and continuous improvement of the shortcomings of LSR methodology and further standardised differences in practice.

Although Elliott et al’s system introduction to the LSR concept described its production schedule,5 it stated that the LSR needed pre-registration and updated roughly once a month. Based on the findings, fewer than half of the current studies have been updated at least once, and many of these did not complete standardised registration or changed their plans midway through the process. Most of the registered research was not reported and updated according to their protocols, and there was no set frequency of updates. Different LSR studies were updated in various forms, such as Article, Review and Letter, and no uniform form has been reached.

Considering the distribution of thematic trends and the current state of the global epidemic, COVID-19 and its virological studies, patient recovery and prognosis will remain a hot topic for LSR research for a long time to come. Although the CENTER-TBI project was launched in 201518 and has been committed to improving the medical care treatment and treatment difference of TBI patients, the thematic outbreak only appeared in 2021 because the coverage renewal strategy of the project replaced previous studies. The update strategy varied considerably between studies. Not all studies retained all the evidence updated in previous dynasties, and some LSRs retain only the evidence of the latest issue, which was also worth our attention. The LNMA and living guidelines have attracted much attention in recent years. We look forward to their further application and dissemination, thus contributing to the continuous progress of EBM.

After carefully reviewing existing studies on LSR, we also raised specific considerations. Since LSR is a process of continuous dynamic renewal and its workload is often substantial, most LSR studies did not mention plans to end or discontinue research. Some studies also failed to explain the advantages of LSR over cumulative MA or whether there is a need for conversion between the two, which may be instructive for further clinical trials. Some LSR studies maintained the same title and coauthor order in historical updates, which may lead readers to believe they were mistakenly the same study. This indicates that LSR studies need to standardise their replacement evolution while maintaining series’ similarity to help distinguish them. As clinical studies are usually undertaken freely, and LSRs are often conducted with a clear purpose, it is difficult to track the differential outcome indicators included in each study, which inevitably leads to a loss or waste of resources in the original study. In traditional SR and MA, the evidence quality is usually assessed using the GRADE Working Group method.52 As for the specificity of the evidence integration of LSR, we also need to determine the quality of its update. However, we are not sure whether the GRADE system is entirely appropriate. In addition, we also mentioned above the issue of the updated presentation form, and we look forward to the refinement and progress of this excellent research method in practical experiments.

To our knowledge, this is the first study to provide a comprehensive bibliometric analysis of the development of the LSR since it was proposed, the current state of LSR-related research, and future perspectives. Based on the comprehensive review and visual presentation, we also put forward some reflections on the research using this methodology. However, there are still some potential limitations in our study. First of all, although we searched the LSR-related studies as comprehensively as possible and made manual supplements, we could still not obtain old versions of a small number of coverage update studies, which might cause differences in literature count, but would not affect our assessment of update frequency. Second, in the author analysis, we tried our best to find the affiliation of similar authors. If they were from the same unit, we added up their contributions. However, we could not tell whether several authors of the same name automatically merged by the software were the same researcher.

Conclusion

This study comprehensively reviewed the related research on LSR and analysed its whole picture and future trend with bibliometrics. Although the history of LSR was not long, it has received much attention and good overall recognition. There was still a wide gap in the attention and application of this research method between countries. Most studies based on the LSR approach were currently focused on COVID-19, and its patient recovery and virological studies may still be the research direction of LSR for a long time to come. The current update of the LSR series of studies varies considerably, and more details need to be further standardised. We look forward to seeing LSR used in more areas in the future.

Handling editor: Soumyadeep Bhaumik
Twitter: @CostaRicazZ
QZ and JX contributed equally.
Contributors: QZ and JX wrote the first draft of the manuscript, did the literature review and systematic search and interpreted the results. JZ and JT were responsible for the study concept and design. LC, LX, JC, MY, GO and HH were responsible for the data collection and collation. QZ, JX, YG and ML conducted data integration and figures. JT, JW and JZ supervised the study. All authors reviewed and approved the final submitted version. JT had final responsibility for the decision to submit for publication.
Funding: This research was partly supported by Gansu Province Science and Technology Plan Funded Project (20CX4ZA027 and 20CX9ZA112). The funders had no role in the study design, collection, analysis or interpretation of the data. The funders did not write the report and had no role in the decision to submit the paper for publication.
Map disclaimer: The inclusion of any map (including the depiction of any boundaries therein), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review: Not commissioned; externally peer-reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Data are available upon request. All Data relevant to the study are included in the article the or uploaded the as Supplemental Information.

Ethics statements

Patient consent for publication:

Ethics approval:

This study did not involve human participants. Data sources for this study were taken from publicly available databases. Ethical approval was not required to be obtained for this study.

Acknowledgements

Helpful comments by anonymous reviewers are acknowledged with thanks.

Akl EA, Haddaway NR, Rada G, et al. Future of evidence ecosystem series: evidence synthesis 2.0: when systematic, scoping, rapid, living, and overviews of reviews come together. J Clin Epidemiol 2020; 123:162–5.
doi:10.1016/j.jclinepi.2020.01.025•Google Scholar •PubMed
Gao Y, Yang K, Cai Y, et al. Updating systematic reviews can improve the precision of outcomes: a comparative study. J Clin Epidemiol 2020; 125:108–19.
doi:10.1016/j.jclinepi.2020.05.019•Google Scholar •PubMed
Garner P, Hopewell S, Chandler J, et al. When and how to update systematic reviews: consensus and checklist. BMJ 2016; 354.
doi:10.1136/bmj.i3507•Google Scholar •PubMed
Shojania KG, Sampson M, Ansari MT, et al. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007; 147:224–33.
doi:10.7326/0003-4819-147-4-200708210-00179•Google Scholar •PubMed
Elliott JH, Synnot A, Turner T, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 2017; 91:23–30.
doi:10.1016/j.jclinepi.2017.08.010•Google Scholar •PubMed
Elliott JH, Turner T, Clavisi O, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med 2014; 11.
doi:10.1371/journal.pmed.1001603•Google Scholar
Tricco AC, Antony J, Zarin W, et al. A scoping review of rapid review methods. BMC Med 2015; 13.
doi:10.1186/s12916-015-0465-6•Google Scholar •PubMed
Akl EA, Meerpohl JJ, Elliott J, et al. Living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol 2017; 91:47–53.
doi:10.1016/j.jclinepi.2017.08.009•Google Scholar •PubMed
Simmonds M, Salanti G, McKenzie J, et al. Living systematic reviews: 3. Statistical methods for updating meta-analyses. J Clin Epidemiol 2017; 91:38–46.
doi:10.1016/j.jclinepi.2017.08.008•Google Scholar •PubMed
Thomas J, Noel-Storr A, Marshall I, et al. Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol 2017; 91:31–7.
doi:10.1016/j.jclinepi.2017.08.011•Google Scholar •PubMed
Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile APP for systematic reviews. Syst Rev 2016; 5.
doi:10.1186/s13643-016-0384-4•Google Scholar •PubMed
van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2010; 84:523–38.
doi:10.1007/s11192-009-0146-3•Google Scholar •PubMed
Gao Y, Ge L, Shi S, et al. Global trends and future prospects of e-waste research: a bibliometric analysis. Environ Sci Pollut Res Int 2019; 26:17809–20.
doi:10.1007/s11356-019-05071-8•Google Scholar •PubMed
Ma D, Yang B, Guan B, et al. A bibliometric analysis of pyroptosis from 2001 to 2021. Front Immunol 2021; 12.
doi:10.3389/fimmu.2021.731933•Google Scholar •PubMed
Aria M, Cuccurullo C. bibliometrix: an R-tool for comprehensive science mapping analysis. J Informetr 2017; 11:959–75.
doi:10.1016/j.joi.2017.08.007•Google Scholar
Elliott JH, Turner T, Clavisi O, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med 2014; 11.
doi:10.1371/journal.pmed.1001603•Google Scholar •PubMed
Synnot A, Gruen RL, Menon D, et al. A new approach to evidence synthesis in traumatic brain injury: a living systematic review. J Neurotrauma 2021; 38:1069–71.
doi:10.1089/neu.2015.4124•Google Scholar •PubMed
Cnossen MC, Scholten AC, Lingsma HF, et al. Adherence to guidelines in adult patients with traumatic brain injury: a living systematic review. J Neurotrauma 2021; 38:1072–85.
doi:10.1089/neu.2015.4121•Google Scholar •PubMed
Zeiler FA, McFadyen C, Newcombe VFJ, et al. Genetic influences on patient-oriented outcomes in traumatic brain injury: a living systematic review of non-apolipoprotein E single-nucleotide polymorphisms. J Neurotrauma 2021; 38:1107–23.
doi:10.1089/neu.2017.5583•Google Scholar •PubMed
Mondello S, Sorinola A, Czeiter E, et al. Blood-based protein biomarkers for the management of traumatic brain injuries in adults presenting to emergency departments with mild brain injury: a living systematic review and meta-analysis. J Neurotrauma 2021; 38:1086–106.
doi:10.1089/neu.2017.5182•Google Scholar •PubMed
McFadyen CA, Zeiler FA, Newcombe V, et al. Apolipoprotein E4 polymorphism and outcomes from traumatic brain injury: a living systematic review and meta-analysis. J Neurotrauma 2021; 38:1124–36.
doi:10.1089/neu.2018.6052•Google Scholar •PubMed
Brazinova A, Rehorcikova V, Taylor MS, et al. Epidemiology of traumatic brain injury in Europe: a living systematic review. J Neurotrauma 2021; 38:1411–40.
doi:10.1089/neu.2015.4126•Google Scholar •PubMed
Créquit P, Chaimani A, Yavchitz A, et al. Comparative efficacy and safety of second-line treatments for advanced non-small cell lung cancer with wild-type or unknown status for epidermal growth factor receptor: a systematic review and network meta-analysis. BMC Med 2017; 15.
doi:10.1186/s12916-017-0954-x•Google Scholar •PubMed
Vogel JP, Dowswell T, Lewin S, et al. Developing and applying a ‘living guidelines’ approach to WHO recommendations on maternal and perinatal health. BMJ Glob Health 2019; 4.
doi:10.1136/bmjgh-2019-001683•Google Scholar •PubMed
Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ 2020; 369.
doi:10.1136/bmj.m1328•Google Scholar •PubMed
Agarwal A, Rochwerg B, Lamontagne F, et al. A living WHO guideline on drugs for covid-19. BMJ 2020; 370.
doi:10.1136/bmj.m3379•Google Scholar •PubMed
Mackey K, King VJ, Gurley S, et al. Risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults: a living systematic review. Ann Intern Med 2020; 173:195–203.
doi:10.7326/M20-1515•Google Scholar •PubMed
Schünemann HJ, Khabsa J, Solo K, et al. Ventilation techniques and risk for transmission of coronavirus disease, including COVID-19: a living systematic review of multiple streams of evidence. Ann Intern Med 2020; 173:204–16.
doi:10.7326/M20-2306•Google Scholar •PubMed
Piechotta V, Chai KL, Valk SJ, et al. Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a living systematic review. Cochrane Database Syst Rev 2020; 7.
doi:10.1002/14651858.CD013600.pub2•Google Scholar •PubMed
Wolfenden L, Wyse RJ, Britton BI, et al. Interventions for increasing fruit and vegetable consumption in preschool aged children. Cochrane Database Syst Rev 2010; 6.
Google Scholar
McRobbie H, Bullen C, Hajek P, et al. Electronic cigarettes for smoking cessation and reduction. Cochrane Database Syst Rev 2012; 11.
Google Scholar
Sbidian E, Le Cleach L, Trinquart L, et al. Systemic pharmacological treatments for chronic plaque psoriasis. Cochrane Database Syst Rev 2015; 2.
Google Scholar
Hodder RK, Stacey FG, Wyse RJ, et al. Interventions for increasing fruit and vegetable consumption in children aged five years and under. Cochrane Database Syst Rev 2017; 9.
doi:10.1002/14651858.CD008552.pub3•Google Scholar •PubMed
Hartmann-Boyce J, McRobbie H, Lindson N, et al. Electronic cigarettes for smoking cessation. Cochrane Database Syst Rev 2020; 10.
doi:10.1002/14651858.CD010216.pub•Google Scholar •PubMed
Sbidian E, Chaimani A, Afach S, et al. Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis. Cochrane Database Syst Rev 2020; 1.
doi:10.1002/14651858.CD011535.pub3•Google Scholar •PubMed
Kansagara D, Mackey K, Vela K, et al. Update alert: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2020; 173.
doi:10.7326/L20-0887•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 2: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2020; 173.
doi:10.7326/L20-0969•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 3: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2020; 173:130–1.
doi:10.7326/L20-1068•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 4: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2020; 173:W147–8.
doi:10.7326/L20-1177•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 5: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2020; 173:167–8.
doi:10.7326/L20-1293•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 6: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2021; 174.
doi:10.7326/L20-1355•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 7: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2021; 174:W25–9.
doi:10.7326/L20-1446•Google Scholar •PubMed
Mackey K, Kansagara D, Vela K, et al. Update alert 8: risks and impact of angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers on SARS-CoV-2 infection in adults. Ann Intern Med 2021; 174:W54–5.
doi:10.7326/L21-0223•Google Scholar •PubMed
Negrini F, de Sire A, Andrenelli E, et al. Rehabilitation and COVID-19: a rapid living systematic review 2020 by Cochrane rehabilitation field. update as of October 31st, 2020. Eur J Phys Rehabil Med 2021; 57:166–70.
doi:10.23736/S1973-9087.20.06723-4•Google Scholar •PubMed
Arce Pardo S, Lai S, Ortiz-Muñoz L, et al. Pulmonary rehabilitation for COVID-19: a living systematic review protocol. Medwave 2021; 21.
doi:10.5867/medwave.2021.06.8223•Google Scholar •PubMed
Winters M, Holden S, Lura CB, et al. Comparative effectiveness of treatments for patellofemoral pain: a living systematic review with network meta-analysis. Br J Sports Med 2020; 55:369–77.
doi:10.1136/bjsports-2020-102819•Google Scholar •PubMed
Agency for Healthcare Research and Quality (US). Living systematic review on cannabis and other plant-based treatments for chronic pain: interim progress reports. Rockville, MD. 2020;
Available: here
Google Scholar
Elliott J, DeJean D, Clifford T, et al. Cannabis for pediatric epilepsy: protocol for a living systematic review. Syst Rev 2018; 7.
doi:10.1186/s13643-018-0761-2•Google Scholar •PubMed
Simmonds M, Elliott JH, Synnot A, et al. Living systematic reviews. Methods Mol Biol 2022; 2345:121–34.
doi:10.1007/978-1-0716-1566-9_7•Google Scholar •PubMed
Piechotta V, Iannizzi C, Chai KL, et al. Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a living systematic review. Cochrane Database Syst Rev 2021; 5.
doi:10.1002/14651858.CD013600.pub4•Google Scholar •PubMed
John A, Eyles E, Webb RT, et al. The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: update of living systematic review. F1000Res 2020; 9.
doi:10.12688/f1000research.25522.1•Google Scholar •PubMed
Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011; 64:383–94.
doi:10.1016/j.jclinepi.2010.04.026•Google Scholar •PubMed

Overview

Abstract
Introduction
Methods
Results
Discussion
Conclusion
References

Supplementary files
Footnotes
Publication history
Rapid Responses

Article metrics

Altmetric

Dimensions