examples of research topics in medicine (e g covid 19)

COVID-19 Research

Stanford Medicine scientists have launched dozens of research projects as part of the global response to COVID-19. Some aim to prevent, diagnose and treat the disease; others aim to understand how it spreads and how people’s immune systems respond to it.

Below is a curated selection, including summaries, of the projects.

To  participate in research ,  browse COVID-19 studies . Our  research registry  also connects people like you with teams conducting  research to make advances in health care. If you are eligible for a study, researchers may contact you to provide additional details on how to participate.

By participating in clinical research, you help accelerate medical science by providing valuable insights into potential treatments and methons of prevention.

Stanford COVID-19 Study Directory Stanford Medicine Research Registry   

To improve our ability to determine who has COVID-19 and treat those infected.

Transmission

To better prevent and understand the transmission of the coronavirus.

Vaccination and Treatment

To improve our ability to prevent COVID-19 and treat those infected.

Epidemiology

To better understand how the coronavirus is spreading.

Data Science and Modeling

To better predict medical, fiscal and resource-related outcomes of the COVID-19 pandemic.

To better understand immune responses to the coronavirus.

Cardiovascular

To better understand the way the virus affects the cardiovascular system.

To better enable the workforce to achieve its goals during the COVID-19 pandemic.

Miscellaneous

A variety of other research projects related to the COVID-19 pandemic.

The list isn’t comprehensive and instead represents a portion of Stanford Medicine research on COVID-19. If you are a Stanford Medicine scientist and would like to see your research included here, please send a note to: [email protected].

The Stanford Institute for Human-Centered Artificial Intelligence has also created a  webpage  for COVID-19 research collaborations and other opportunities, such as research positions, internships and funding. If you would like to submit an opening please use the following  form  and they will post it on their website.

Support Stanford Medicine’s response to COVID-19 by  making a gift .

COVID-19 Research Projects

The COVID-19 research landscape: Measuring topics and collaborations using scientific literature

Affiliations.

• 1 Institute of Medical Information, Chinese Academy of Medical Sciences.
• 2 Digital China Health Technologies Co. Ltd., Beijing, China.
• PMID: 33120818
• PMCID: PMC7581087
• DOI: 10.1097/MD.0000000000022849

Objectives: The Coronavirus Disease 2019 (COVID-19) caused heavy burdens and brought tremendous challenges to global public health. This study aimed to investigate collaboration relationships, research topics, and research trends on COVID-19 using scientific literature.

Method: COVID-19-related articles published from January 1 to July 1, 2020 were retrieved from PubMed database. A total of 27,370 articles were included. Excel 2010, Medical Text Indexer (MTI), VOSviewer, and D3.js were used to summarize bibliometric features.

Results: The number of the COVID-19 research publications has been continuously increasing after its break. United States was the most productive and active country for COVID-19 research, with the largest number of publications and collaboration relationships. Huazhong University of Science and Technology from China was the most productive institute on the number of publications, and University of Toronto from Canada ranked as Top 1 institute for global research collaboration. Four key research topics were identified, of which the topic of epidemiology and public health interventions has gathered highest attentions. Topic of virus infection and immunity has been more focused during the early stage of COVID-19 outbreak compared with later stage. The topic popularity of clinical symptoms and diagnosis has been steady.

Conclusions: Our topic analysis results revealed that the study of drug treatment was insufficient. To achieve critical breakthroughs of this research area, more interdisciplinary, multi-institutional, and global research collaborations are needed.

Publication types

• Systematic Review
• Betacoronavirus*
• Bibliometrics*
• Biomedical Research / trends*
• Coronavirus Infections* / diagnosis
• Coronavirus Infections* / epidemiology
• Coronavirus Infections* / therapy
• Global Health
• Intersectoral Collaboration
• Pneumonia, Viral* / diagnosis
• Pneumonia, Viral* / epidemiology
• Pneumonia, Viral* / therapy
• Publishing / trends*
• Research Design / trends

Marie Skłodowska-Curie Actions

Projects researching covid–19, sars-cov-2 and related topics.

The current COVID-19 outbreak has not caught EU-funded research off guard. The Marie Skłodowska-Curie Actions (MSCA) of the European Commission are supporting outstanding researchers in finding solutions to challenges posed by the novel coronavirus disease COVID-19 and other infectious diseases.

This page will be regularly updated with MSCA projects, results and testimonials relevant to COVID-19, SARS-CoV-2 and related topics.

DIAGNOSTICS AND TREATMENTS (including vaccines)

Diabetic nephropathy modelling in hesc-derived 3d kidney organoids.

EPIORGABOLISM is studying how SARS-Co-V2, the coronavirus responsible for the 2019 novel coronavirus disease (COVID-19), interacts with and infects kidney cells. Together with the lung, the kidney is one of the main organs affected by the COVID-19 disease. Dr Carmen Hurtado, researcher of EPIORGABOLISM, is currently generating human kidney organoids from human pluripotent stem cells.

The use of human organoids allows to test treatments against coronavirus in an agile way, dramatically reducing the time human drug trials take. Hurtado is part of international research team has identified a drug capable of blocking the effects of the SARS-CoV-2 virus. The findings have been partially supported by EPIORGABOLISM and published in the journal ‘Cell’.

Find out more

• Trial drug shows promise in fighting coronavirus
• Watch the testimonial of Carmen Hurtado , researcher of the EPIORGABOLISM project.

Host switching pathogens, infectious outbreaks and zoonosis; a Marie Sklodowska-Curie Training Network

HONOURs is teaching 15 talented young researchers, including coronavirologists, to become “preparedness-experts”. The project involves 11 laboratories, all at the forefront of novel virus investigations and characterizations. HONOUR reacted in January 2020, immediately after the emergence of COVID-19, by starting work on SARS-CoV-2. A synthetic biology virus culture system was developed to swiftly evaluate therapy options, next to rapid tests to determine virus shedding on location. The quality of protective immunity was evaluated, and a search started on the most suitable animal model to battle the virus and provide therapy options. HONOURs is devoting its expert knowledge to fight this coronavirus and provide therapy options.

• HONOURs: Virus Outbreak Preparedness and COVID-19
• Visit the HONOURs website

INnate-ImmunomeTabolIsm as Antiviral TargEt

The global COVID-19 pandemic highlights an urgent need for innovation in the development of novel antiviral strategies and therapies. INITIATE has recruited 15 young PhD candidates to become experts in the field of antiviral immunometabolism, with a focus on RNA viruses – including coronaviruses. While it is clear that viral replication, metabolic pathways, and host immune responses are tightly interconnected, the host molecular pathways that impact viral pathogenesis are not well-defined. With the emergence of COVID-19, eight of the INITIATE projects have included SARS-CoV-2 in their research programs to understand coronavirus molecular virology, the role of the host immune response in driving COVID-19 immunopathogenesis and the potential of targeting host metabolism as therapeutic strategies.

Organoids for Virus Research - An innovative training-ETN programme

ORGANOVIR  is contributing to COVID-19 research in a variety of ways, and several of its researchers are currently working on the development of new antivirals to combat the disease. Researchers at KU Leuven (Belgium) are studying the way in which coronaviruses evolve, and are searching out possible targets for further remedies. The project also investigating active substances – or a combination of them – in existing medicines that could be effective against SARS-CoV-2. ORGANOVIR is also conducting pre-clinical tests for a vaccine against COVID-19 using a technology based on the yellow fever vaccine.

In parallel, a group of researchers at the Jagiellonian University (Poland) is studying the infection on the single-cell and tissue level in different organs and cell types, working on virus inhibitors and collaborating with companies to create a point of care diagnostics based on different platforms. The group is also studying the course of the pandemic in Poland and monitoring the virus variability in the country.

ORGANOVIR’s coordinators have been intensively working on clinical and diagnostic tasks and set up new COVID-19 research at the Amsterdam UMC (The Netherlands). This has resulted in the launch of COVID-KIDS, a study on immunity in children, and the use of 3D culture models for COVID-19 studies.

• Read about the COVID-19 activities of ORGANOVIR partners
• Read the testimonial of Mariana Guedes , researcher for the OrganoVir project
• Read the testimonial of Thuc Do , researcher for the OrganoVir project
• Air-liquid interface cultures of nasal epithelial cells to investigate factors critical for viral entry into host cells

MECHANISMS OF INFECTION, IMMUNE REACTIONS AND HOST-PATHAGEN INTERACTION

Unravelling species barriers of coronaviruses.

COV RESTRIC  targeted the precise mechanisms that allow coronaviruses to jump across species. Dr Stephanie Pfänder, researcher of COV RESTRIC, worked on various virological aspects of emerging viruses – with a focus on emerging coronaviruses. Her work has the potential to lead to novel strategies to protect cells against coronavirus infection. This is crucial to fight the COVID-19 pandemic – and to help insulate society against future coronavirus outbreaks.

• Read the testimonial of Stephanie Pfänder , researcher of the COV RESTRIC project.
• Host proteins involved in species barriers of viral infections

DIGITAL TOOLS, DATA AND MODELLING

Research and innovation staff exchange network of european data scientists.

The NeEDS  consortium is currently focusing on the emerging data challenges that come with the COVID-19 pandemic. In Spain, the first cases of the COVID-19 pandemic were confirmed late February 2020 and data started to be collected daily by the different regions. Data and Data Science tools turned out to be crucial to assist decision makers in this highly uncertain context. NeEDS and the scientific collaborations they enjoy were fundamental to create a working group of data scientists from different European universities, which has developed an Artificial Intelligence tool to provide short-term predictions of the pandemic’s evolution. With this novel methodology, NeEDS as contributed to the cooperative efforts coordinated by the Spanish Commission of Mathematics to support data-driven decision making related to the COVID19 pandemic. In a recent interview , Project Coordinator Dolores Romero Morales has reflected on the potential of the NeEDS expertise and the efforts of tackling these data challenges within the team. The consortium is tackling other important Data Science questions, e.g., using spatial data to support COVID19 information apps or addressing the pressing data privacy needs.

• Read about the COVID-19 activities of NeEDS and its partners
• On Sparse Ensemble Methods: An Application to Short-Term Predictions of the Evolution of COVID-19
• Read the testimonials of Remedios Sillero, Cristina Molero and Sandra Benitez , seconded researchers for the NeEDS project.

Pan-genome Graph Algorithms and Data Integration

Researchers involved in PANGAIA  are investigating how massive amounts of genome sequence data can be ordered and analysed for their use in biomedicine. Their work has important implications in areas such as bacteria and virus research, investigation of drug resistance mechanisms and vaccine development: big data technology can help to identify the characteristics of new strains of viruses such as SARS-CoV-2 and bacteria by comparing their genomes.

• Identifying large data sets to help coronavirus research
• Identifying pathogenic genes in virus strains at a glance

Modelling Infectious Diseases in Dynamic, relocated, refugee populations

In order to assist policy-makers in mitigating outbreaks, MIDIDP  has created realistic models to simulate the spread of infectious diseases in under-vaccinated refugee populations in Europe and neighbouring countries. Dr Hasan Güçlü, researcher of MIDIDP, has created a model that simulates the spread of COVID-19 in populations with variable demographics.

• Read the testimonial of Hasan Güçlü , researcher of the MIDIDP project.

PUBLIC HEALTH, PREPAREDNESS AND RESPONSE

Disability and disease during the 1918 influenza pandemic: implications for preparedness policies.

As the current COVID-19 pandemic shows, people with disabilities are at increased risk for complications and death as they are often neglected in epidemic responses. Dr Jessica Dimka, researcher of DIS2 , is exploring disability as a risk factor in pandemics. Using the 1918 Spanish influenza pandemic as a model, the project seeks to promote more equitable public health plans and interventions. Dimka points out that people with disabilities must be considered in all pandemic strategies: their lives, livelihoods and rights are not expendable.

• Read the testimonial of Jessica Dimka , researcher of the DIS2 project.

MULTIDISCIPLINARY PROJECTS

Protecting human rights and public health in global pandemics.

THEMIS is an interdisciplinary research project that reacts to the increasing occurrence of global pandemics, like the caused by the present COVID-19 disease, and restrictive public health measures taken to respond to these threats. Using a rights-based approach, Dr Patrycja Dąbrowska-Kłosińska, researcher of THEMIS, intends to create a better understanding of how to prepare for, and respond to, global pandemics.

The project seeks to offer a vital reference for policy-making at national, regional and global levels – one that prioritises fair pandemic preparedness to cross-border health threats. The project has offered critical guidance during the current COVID-19 pandemic, which has required a previously unimagined scale of coordinated, public health-control measures as well as consideration of human-rights implications worldwide.

• Read the testimonial of Patrycja Dąbrowska-Kłosińska , researcher of the THEMIS project.

Martí I Franquès COFUND

Since the emergence of COVID-19, several fellows involved in the Martí Franquès Programme (MFP) have been working on solutions to the current crisis. Researchers are developing an epidemiological mathematical model that infers the status of the epidemic, thereby monitoring and estimating the impact of interventions on the spread of COVID-19.

In parallel, another group of researchers is implementing an original virtual screening protocol to reposition approved drugs. This would allow predicting which of them could inhibit the main protease of the virus (M-pro), a key target for antiviral drugs given its essential role in the virus replication.

• Read the testimonial of Benjamin Steinegger , whose research is developing a mathematical framework to monitor and estimate the impact of interventions on the COVID-19 pandemic.

Project outcomes

• Modelling the impact of interventions on the spread of COVID-19
• Prediction of novel inhibitors of the main protease of SARS-CoV-2
• See all the results relevant to COVID-19 produced by MFP fellows

The launch of a new industrial PhD programme at EPFL

Several fellows involved in the EPFLinnovators  project are working on solutions to COVID-19 since the start of the crisis. Teams of researchers are developing subunit vaccines against the SARS-CoV-2 virus, investigating the potential use of cyclodextrin derivatives to prevent and treat the infections caused by SARS-CoV-2, and analysing the mechanical aspects of SARS-CoV-2 entry into cells.

• Read the testimonial of Xiaomeng Hu , researcher of the EPFLInnovators project.
• Subunit vaccines against SARS-CoV-2
• Non-toxic cyclodextrin derivative against viruses at micromolar concentration
• Variations in clathrin mediated endocytosis on a mammalian cell membrane

SOCIAL BEHAVIOUR AND IMPACT

Leading fellows.

Over the last decade, the reliance on online products and services has steadily increased, but since the beginning of the COVID-19 pandemic it has escalated to an unprecedented level. Dr Matthew Dennis, researcher of the LEaDing Fellows COFUND project at TU Delft (the Netherlands), examines the ethical implications and value trade-offs as societies attempt to transition across the digital divide. His project highlights that an ethical reflection on this digital transition is urgently needed, as digital solutions to problems generated by COVID-19 may create winners and losers – likely disproportionately affecting vulnerable users. By addressing these issues, the pandemic may foster the kind of social and political interconnectedness that was envisioned at the start of the crisis.

• Read the testimonial of Matthew James Dennis , researcher of the LEaDing Fellows project.

MSCA on social media

The MSCA social media are continuously updated with testimonials of MSCA fellows, supervisors, coordinators and projects working to find solutions to challenges posed by COVID-19 and other infectious diseases.

• MSCA on Twitter
• MSCA Facebook page

Related content

Related information, thanks for your feedback.

We are happy to see that your experience was positive. Don't forget to share the pages you like with your friends and colleagues.

If you need to ask a question, please contact Europe direct .

• Research article
• Open access
• Published: 04 June 2021

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

• Israel Júnior Borges do Nascimento 1 , 2 ,
• Dónal P. O’Mathúna 3 , 4 ,
• Thilo Caspar von Groote 5 ,
• Hebatullah Mohamed Abdulazeem 6 ,
• Ishanka Weerasekara 7 , 8 ,
• Ana Marusic 9 ,
• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 10 ,
• Vinicius Tassoni Civile 11 ,
• Irena Zakarija-Grkovic 9 ,
• Tina Poklepovic Pericic 9 ,
• Alvaro Nagib Atallah 11 ,
• Santino Filoso 12 ,
• Nicola Luigi Bragazzi 13 &
• Milena Soriano Marcolino 1

On behalf of the International Network of Coronavirus Disease 2019 (InterNetCOVID-19)

BMC Infectious Diseases volume  21 , Article number:  525 ( 2021 ) Cite this article

16k Accesses

29 Citations

13 Altmetric

Metrics details

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as “critically low”. Identified symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

Conclusions

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Peer Review reports

The spread of the “Severe Acute Respiratory Coronavirus 2” (SARS-CoV-2), the causal agent of COVID-19, was characterized as a pandemic by the World Health Organization (WHO) in March 2020 and has triggered an international public health emergency [ 1 ]. The numbers of confirmed cases and deaths due to COVID-19 are rapidly escalating, counting in millions [ 2 ], causing massive economic strain, and escalating healthcare and public health expenses [ 3 , 4 ].

The research community has responded by publishing an impressive number of scientific reports related to COVID-19. The world was alerted to the new disease at the beginning of 2020 [ 1 ], and by mid-March 2020, more than 2000 articles had been published on COVID-19 in scholarly journals, with 25% of them containing original data [ 5 ]. The living map of COVID-19 evidence, curated by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), contained more than 40,000 records by February 2021 [ 6 ]. More than 100,000 records on PubMed were labeled as “SARS-CoV-2 literature, sequence, and clinical content” by February 2021 [ 7 ].

Due to publication speed, the research community has voiced concerns regarding the quality and reproducibility of evidence produced during the COVID-19 pandemic, warning of the potential damaging approach of “publish first, retract later” [ 8 ]. It appears that these concerns are not unfounded, as it has been reported that COVID-19 articles were overrepresented in the pool of retracted articles in 2020 [ 9 ]. These concerns about inadequate evidence are of major importance because they can lead to poor clinical practice and inappropriate policies [ 10 ].

Systematic reviews are a cornerstone of today’s evidence-informed decision-making. By synthesizing all relevant evidence regarding a particular topic, systematic reviews reflect the current scientific knowledge. Systematic reviews are considered to be at the highest level in the hierarchy of evidence and should be used to make informed decisions. However, with high numbers of systematic reviews of different scope and methodological quality being published, overviews of multiple systematic reviews that assess their methodological quality are essential [ 11 , 12 , 13 ]. An overview of systematic reviews helps identify and organize the literature and highlights areas of priority in decision-making.

In this overview of systematic reviews, we aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Methodology

Research question.

This overview’s primary objective was to summarize and critically appraise systematic reviews that assessed any type of primary clinical data from patients infected with SARS-CoV-2. Our research question was purposefully broad because we wanted to analyze as many systematic reviews as possible that were available early following the COVID-19 outbreak.

Study design

We conducted an overview of systematic reviews. The idea for this overview originated in a protocol for a systematic review submitted to PROSPERO (CRD42020170623), which indicated a plan to conduct an overview.

Overviews of systematic reviews use explicit and systematic methods for searching and identifying multiple systematic reviews addressing related research questions in the same field to extract and analyze evidence across important outcomes. Overviews of systematic reviews are in principle similar to systematic reviews of interventions, but the unit of analysis is a systematic review [ 14 , 15 , 16 ].

We used the overview methodology instead of other evidence synthesis methods to allow us to collate and appraise multiple systematic reviews on this topic, and to extract and analyze their results across relevant topics [ 17 ]. The overview and meta-analysis of systematic reviews allowed us to investigate the methodological quality of included studies, summarize results, and identify specific areas of available or limited evidence, thereby strengthening the current understanding of this novel disease and guiding future research [ 13 ].

A reporting guideline for overviews of reviews is currently under development, i.e., Preferred Reporting Items for Overviews of Reviews (PRIOR) [ 18 ]. As the PRIOR checklist is still not published, this study was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 statement [ 19 ]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [ 14 ].

Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.

Eligibility criteria

Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19 including, but not limited to, those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings, or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.

No restrictions were imposed on the design of the primary studies included within the systematic reviews, the last search date, whether the review included meta-analyses or language. Reviews related to SARS-CoV-2 and other coronaviruses were eligible, but from those reviews, we analyzed only data related to SARS-CoV-2.

No consensus definition exists for a systematic review [ 20 ], and debates continue about the defining characteristics of a systematic review [ 21 ]. Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion [ 14 ]. That is supported by a recent scoping review about guidance for overviews of systematic reviews [ 22 ].

Thus, for this study, we defined a systematic review as a research report which searched for primary research studies on a specific topic using an explicit search strategy, had a detailed description of the methods with explicit inclusion criteria provided, and provided a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Cochrane and non-Cochrane systematic reviews were considered eligible for inclusion, with or without meta-analysis, and regardless of the study design, language restriction and methodology of the included primary studies. To be eligible for inclusion, reviews had to be clearly analyzing data related to SARS-CoV-2 (associated or not with other viruses). We excluded narrative reviews without those characteristics as these are less likely to be replicable and are more prone to bias.

Scoping reviews and rapid reviews were eligible for inclusion in this overview if they met our pre-defined inclusion criteria noted above. We included reviews that addressed SARS-CoV-2 and other coronaviruses if they reported separate data regarding SARS-CoV-2.

Information sources

Nine databases were searched for eligible records published between December 1, 2019, and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.

The comprehensive search strategy for each database is provided in Additional file 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported into the Covidence platform [ 23 ]. In addition to database searches, we screened reference lists of reviews included after screening records retrieved via databases.

Study selection

All searches, screening of titles and abstracts, and record selection, were performed independently by two investigators using the Covidence platform [ 23 ]. Articles deemed potentially eligible were retrieved for full-text screening carried out independently by two investigators. Discrepancies at all stages were resolved by consensus. During the screening, records published in languages other than English were translated by a native/fluent speaker.

Data collection process

We custom designed a data extraction table for this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.

We extracted the following data: article identification data (authors’ name and journal of publication), search period, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA), we extracted journal rank (quartile) and Journal Impact Factor (JIF).

We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no), and length of stay in the intensive care unit.

The following outcomes were categorized as exploratory: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).

We also collected data on reporting guidelines and requirements for the publication of systematic reviews and meta-analyses from journal websites where included reviews were published.

Quality assessment in individual reviews

Two researchers independently assessed the reviews’ quality using the “A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2)”. We acknowledge that the AMSTAR 2 was created as “a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both” [ 24 ]. However, since AMSTAR 2 was designed for systematic reviews of intervention trials, and we included additional types of systematic reviews, we adjusted some AMSTAR 2 ratings and reported these in Additional file 2 .

Adherence to each item was rated as follows: yes, partial yes, no, or not applicable (such as when a meta-analysis was not conducted). The overall confidence in the results of the review is rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains, which are items 2, 4, 7, 9, 11, 13, 15 as defined by AMSTAR 2 authors [ 24 ]. We reported our adherence ratings for transparency of our decision with accompanying explanations, for each item, in each included review.

One of the included systematic reviews was conducted by some members of this author team [ 25 ]. This review was initially assessed independently by two authors who were not co-authors of that review to prevent the risk of bias in assessing this study.

Synthesis of results

For data synthesis, we prepared a table summarizing each systematic review. Graphs illustrating the mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths, and limitations.

For analysis of the prevalence of clinical outcomes, we extracted data on the number of events and the total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9–6), using the “metaprop” function for reviews that did not perform a meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform a meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ 2 . We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. We created forest plots using the RStudio© software, with the “metafor” package (version 2.1–0) and “forest” function.

Managing overlapping systematic reviews

Some of the included systematic reviews that address the same or similar research questions may include the same primary studies in overviews. Including such overlapping reviews may introduce bias when outcome data from the same primary study are included in the analyses of an overview multiple times. Thus, in summaries of evidence, multiple-counting of the same outcome data will give data from some primary studies too much influence [ 14 ]. In this overview, we did not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic [ 14 ]. To avoid any bias in summary estimates associated with overlapping reviews, we generated forest plots showing data from individual systematic reviews, but the results were not pooled because some primary studies were included in multiple reviews.

Our search retrieved 1063 publications, of which 175 were duplicates. Most publications were excluded after the title and abstract analysis ( n = 860). Among the 28 studies selected for full-text screening, 10 were excluded for the reasons described in Additional file 3 , and 18 were included in the final analysis (Fig. 1 ) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Reference list screening did not retrieve any additional systematic reviews.

PRISMA flow diagram

Characteristics of included reviews

Summary features of 18 systematic reviews are presented in Table 1 . They were published in 14 different journals. Only four of these journals had specific requirements for systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology, and Clinical Research in Cardiology . Two journals reported that they published only invited reviews ( Journal of Medical Virology and Clinica Chimica Acta ). Three systematic reviews in our study were published as letters; one was labeled as a scoping review and another as a rapid review (Table 2 ).

All reviews were published in English, in first quartile (Q1) journals, with JIF ranging from 1.692 to 6.062. One review was empty, meaning that its search did not identify any relevant studies; i.e., no primary studies were included [ 36 ]. The remaining 17 reviews included 269 unique studies; the majority ( N = 211; 78%) were included in only a single review included in our study (range: 1 to 12). Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts, and other observational studies. We found only one review that included randomized clinical trials [ 38 ]. In the included reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020. Ten systematic reviews included meta-analyses. The list of primary studies found in the included systematic reviews is shown in Additional file 4 , as well as the number of reviews in which each primary study was included.

Population and study designs

Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS), or any other correlated complication. One review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [ 36 ], one review was focused on pediatric patients [ 34 ], and one review investigated COVID-19 in pregnant women [ 37 ]. Most reviews assessed clinical symptoms, laboratory findings, or radiological results.

Systematic review findings

The summary of findings from individual reviews is shown in Table 2 . Overall, all-cause mortality ranged from 0.3 to 13.9% (Fig. 2 ).

A meta-analysis of the prevalence of mortality

Clinical symptoms

Seven reviews described the main clinical manifestations of COVID-19 [ 26 , 28 , 29 , 34 , 35 , 39 , 41 ]. Three of them provided only a narrative discussion of symptoms [ 26 , 34 , 35 ]. In the reviews that performed a statistical analysis of the incidence of different clinical symptoms, symptoms in patients with COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figs. 3 , 4 , 5 , 6 , 7 , 8 and 9 ). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).

A meta-analysis of the prevalence of fever

A meta-analysis of the prevalence of cough

A meta-analysis of the prevalence of dyspnea

A meta-analysis of the prevalence of fatigue or myalgia

A meta-analysis of the prevalence of headache

A meta-analysis of the prevalence of gastrointestinal disorders

A meta-analysis of the prevalence of sore throat

Diagnostic aspects

Three reviews described methodologies, protocols, and tools used for establishing the diagnosis of COVID-19 [ 26 , 34 , 38 ]. The use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic method mentioned in the included studies. These diagnostic tests have been widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study with clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least two symptoms suggestive of COVID-19, together with laboratory and chest radiography abnormalities) [ 34 ].

Therapeutic possibilities

Pharmacological and non-pharmacological interventions (supportive therapies) used in treating patients with COVID-19 were reported in five reviews [ 25 , 27 , 34 , 35 , 38 ]. Antivirals used empirically for COVID-19 treatment were reported in seven reviews [ 25 , 27 , 34 , 35 , 37 , 38 , 41 ]; most commonly used were protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analogs (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in two studies [ 25 , 35 ]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation) [ 25 ]. The use of antibiotics, both empirically and to treat secondary pneumonia, was reported in six studies [ 25 , 26 , 27 , 34 , 35 , 38 ]. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [ 27 ]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. The use of mesenchymal stem cells, antifungals, and glucocorticoids were described in four reviews [ 25 , 34 , 35 , 38 ].

Laboratory and radiological findings

Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [ 25 , 29 , 30 , 32 , 33 , 34 , 35 , 39 ]; elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described in those eight reviews. Lippi et al. assessed cardiac troponin I (cTnI) [ 25 ], procalcitonin [ 32 ], and platelet count [ 33 ] in COVID-19 patients. Elevated levels of procalcitonin [ 32 ] and cTnI [ 30 ] were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections [ 33 ].

Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [ 25 , 34 , 35 , 39 , 40 , 41 ]. Those six reviews showed that septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs, and pneumothorax were observed in patients suffering from COVID-19.

Quality of evidence in individual systematic reviews

Table 3 shows the detailed results of the quality assessment of 18 systematic reviews, including the assessment of individual items and summary assessment. A detailed explanation for each decision in each review is available in Additional file 5 .

Using AMSTAR 2 criteria, confidence in the results of all 18 reviews was rated as “critically low” (Table 3 ). Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of an explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).

Risk of bias assessment, based on a reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method, were reported only in one review [ 25 ]. Five reviews presented a table summarizing bias, using various risk of bias tools [ 25 , 29 , 39 , 40 , 41 ]. One review analyzed “study quality” [ 37 ]. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis [ 28 ].

This overview of systematic reviews analyzed the first 18 systematic reviews published after the onset of the COVID-19 pandemic, up to March 24, 2020, with primary studies involving more than 60,000 patients. Using AMSTAR-2, we judged that our confidence in all those reviews was “critically low”. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings, and interventions. We found no systematic reviews on the utility of diagnostic tests.

Symptoms were reported in seven reviews; most of the patients had a fever, cough, dyspnea, myalgia or muscle fatigue, and gastrointestinal disorders such as diarrhea, nausea, or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been described in patients infected with COVID-19 [ 43 ]; however, this was not reported in any of the reviews included in this overview. During the SARS outbreak in 2002, there were reports of impairment of the sense of smell associated with the disease [ 44 , 45 ].

The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control, and treatment facilities; variability in the mortality rate fits the pattern of emerging infectious diseases [ 46 ]. Furthermore, the reported cases did not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries had limited access to diagnostic tests or have implemented testing policies later than the others. Considering the lack of reviews assessing diagnostic testing (sensitivity, specificity, and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, considerable imprecision around the calculated mortality rates existed in the early stage of the COVID-19 pandemic.

Few reviews included treatment data. Those reviews described studies considered to be at a very low level of evidence: usually small, retrospective studies with very heterogeneous populations. Seven reviews analyzed laboratory parameters; those reviews could have been useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, such as assessing which patients need to be reassessed more frequently.

All systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Most of the original studies included in the reviews were case series and case reports, impacting the quality of evidence. Such evidence has major implications for clinical practice and the use of these reviews in evidence-based practice and policy. Clinicians, patients, and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.

We acknowledge that there are numerous challenges associated with analyzing COVID-19 data during a pandemic [ 47 ]. High-quality evidence syntheses are needed for decision-making, but each type of evidence syntheses is associated with its inherent challenges.

The creation of classic systematic reviews requires considerable time and effort; with massive research output, they quickly become outdated, and preparing updated versions also requires considerable time. A recent study showed that updates of non-Cochrane systematic reviews are published a median of 5 years after the publication of the previous version [ 48 ].

Authors may register a review and then abandon it [ 49 ], but the existence of a public record that is not updated may lead other authors to believe that the review is still ongoing. A quarter of Cochrane review protocols remains unpublished as completed systematic reviews 8 years after protocol publication [ 50 ].

Rapid reviews can be used to summarize the evidence, but they involve methodological sacrifices and simplifications to produce information promptly, with inconsistent methodological approaches [ 51 ]. However, rapid reviews are justified in times of public health emergencies, and even Cochrane has resorted to publishing rapid reviews in response to the COVID-19 crisis [ 52 ]. Rapid reviews were eligible for inclusion in this overview, but only one of the 18 reviews included in this study was labeled as a rapid review.

Ideally, COVID-19 evidence would be continually summarized in a series of high-quality living systematic reviews, types of evidence synthesis defined as “ a systematic review which is continually updated, incorporating relevant new evidence as it becomes available ” [ 53 ]. However, conducting living systematic reviews requires considerable resources, calling into question the sustainability of such evidence synthesis over long periods [ 54 ].

Research reports about COVID-19 will contribute to research waste if they are poorly designed, poorly reported, or simply not necessary. In principle, systematic reviews should help reduce research waste as they usually provide recommendations for further research that is needed or may advise that sufficient evidence exists on a particular topic [ 55 ]. However, systematic reviews can also contribute to growing research waste when they are not needed, or poorly conducted and reported. Our present study clearly shows that most of the systematic reviews that were published early on in the COVID-19 pandemic could be categorized as research waste, as our confidence in their results is critically low.

Our study has some limitations. One is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. In three reviews, the methodological quality appraisal was challenging because they were published as letters, or labeled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as “not reported”, resulting in low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low-quality review methods. To make their review available more rapidly and more concisely, the authors may have omitted methodological details. A general issue during a crisis is that speed and completeness must be balanced. However, maintaining high standards requires proper resourcing and commitment to ensure that the users of systematic reviews can have high confidence in the results.

Furthermore, we used adjusted AMSTAR 2 scoring, as the tool was designed for critical appraisal of reviews of interventions. Some reviews may have received lower scores than actually warranted in spite of these adjustments.

Another limitation of our study may be the inclusion of multiple overlapping reviews, as some included reviews included the same primary studies. According to the Cochrane Handbook, including overlapping reviews may be appropriate when the review’s aim is “ to present and describe the current body of systematic review evidence on a topic ” [ 12 ], which was our aim. To avoid bias with summarizing evidence from overlapping reviews, we presented the forest plots without summary estimates. The forest plots serve to inform readers about the effect sizes for outcomes that were reported in each review.

Several authors from this study have contributed to one of the reviews identified [ 25 ]. To reduce the risk of any bias, two authors who did not co-author the review in question initially assessed its quality and limitations.

Finally, we note that the systematic reviews included in our overview may have had issues that our analysis did not identify because we did not analyze their primary studies to verify the accuracy of the data and information they presented. We give two examples to substantiate this possibility. Lovato et al. wrote a commentary on the review of Sun et al. [ 41 ], in which they criticized the authors’ conclusion that sore throat is rare in COVID-19 patients [ 56 ]. Lovato et al. highlighted that multiple studies included in Sun et al. did not accurately describe participants’ clinical presentations, warning that only three studies clearly reported data on sore throat [ 56 ].

In another example, Leung [ 57 ] warned about the review of Li, L.Q. et al. [ 29 ]: “ it is possible that this statistic was computed using overlapped samples, therefore some patients were double counted ”. Li et al. responded to Leung that it is uncertain whether the data overlapped, as they used data from published articles and did not have access to the original data; they also reported that they requested original data and that they plan to re-do their analyses once they receive them; they also urged readers to treat the data with caution [ 58 ]. This points to the evolving nature of evidence during a crisis.

Our study’s strength is that this overview adds to the current knowledge by providing a comprehensive summary of all the evidence synthesis about COVID-19 available early after the onset of the pandemic. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews.

In conclusion, in this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all the reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic could be categorized as research waste. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards to provide patients, clinicians, and decision-makers trustworthy evidence.

Availability of data and materials

All data collected and analyzed within this study are available from the corresponding author on reasonable request.

World Health Organization. Timeline - COVID-19: Available at: https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 1 June 2021.

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available at: https://coronavirus.jhu.edu/map.html . Accessed 1 June 2021.

Anzai A, Kobayashi T, Linton NM, Kinoshita R, Hayashi K, Suzuki A, et al. Assessing the Impact of Reduced Travel on Exportation Dynamics of Novel Coronavirus Infection (COVID-19). J Clin Med. 2020;9(2):601.

Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020;368(6489):395–400. https://doi.org/10.1126/science.aba9757 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fidahic M, Nujic D, Runjic R, Civljak M, Markotic F, Lovric Makaric Z, et al. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med Res Methodol. 2020;20(1):161. https://doi.org/10.1186/s12874-020-01047-2 .

EPPI Centre . COVID-19: a living systematic map of the evidence. Available at: http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx . Accessed 1 June 2021.

NCBI SARS-CoV-2 Resources. Available at: https://www.ncbi.nlm.nih.gov/sars-cov-2/ . Accessed 1 June 2021.

Gustot T. Quality and reproducibility during the COVID-19 pandemic. JHEP Rep. 2020;2(4):100141. https://doi.org/10.1016/j.jhepr.2020.100141 .

Article   PubMed   PubMed Central   Google Scholar  

Kodvanj, I., et al., Publishing of COVID-19 Preprints in Peer-reviewed Journals, Preprinting Trends, Public Discussion and Quality Issues. Preprint article. bioRxiv 2020.11.23.394577; doi: https://doi.org/10.1101/2020.11.23.394577 .

Dobler CC. Poor quality research and clinical practice during COVID-19. Breathe (Sheff). 2020;16(2):200112. https://doi.org/10.1183/20734735.0112-2020 .

Article   Google Scholar  

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 .

Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/10.1186/s13643-017-0617-1 .

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. 2020. Available from www.training.cochrane.org/handbook .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.1 (updated September 2020). Cochrane. 2020; Available from www.training.cochrane.org/handbook .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18. https://doi.org/10.1186/s13643-018-0914-3 .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):29. https://doi.org/10.1186/s13643-018-0768-8 .

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/10.1186/s13643-018-0695-8 .

Pollock M, Fernandes RM, Pieper D, Tricco AC, Gates M, Gates A, et al. Preferred reporting items for overviews of reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):335. https://doi.org/10.1186/s13643-019-1252-9 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0 .

Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5. https://doi.org/10.1016/j.jclinepi.2017.08.002 .

Article   PubMed   Google Scholar  

Gates M, Gates A, Guitard S, Pollock M, Hartling L. Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review. Syst Rev. 2020;9(1):254. https://doi.org/10.1186/s13643-020-01509-0 .

Covidence - systematic review software. Available at: https://www.covidence.org/ . Accessed 1 June 2021.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Borges do Nascimento IJ, et al. Novel Coronavirus Infection (COVID-19) in Humans: A Scoping Review and Meta-Analysis. J Clin Med. 2020;9(4):941.

Article   PubMed Central   Google Scholar  

Adhikari SP, Meng S, Wu YJ, Mao YP, Ye RX, Wang QZ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty. 2020;9(1):29. https://doi.org/10.1186/s40249-020-00646-x .

Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020;57:279–83. https://doi.org/10.1016/j.jcrc.2020.03.005 .

Li B, Yang J, Zhao F, Zhi L, Wang X, Liu L, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol. 2020;109(5):531–8. https://doi.org/10.1007/s00392-020-01626-9 .

Article   CAS   PubMed   Google Scholar  

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(6):577–83. https://doi.org/10.1002/jmv.25757 .

Lippi G, Lavie CJ, Sanchis-Gomar F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): evidence from a meta-analysis. Prog Cardiovasc Dis. 2020;63(3):390–1. https://doi.org/10.1016/j.pcad.2020.03.001 .

Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107–8. https://doi.org/10.1016/j.ejim.2020.03.014 .

Lippi G, Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–1. https://doi.org/10.1016/j.cca.2020.03.004 .

Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145–8. https://doi.org/10.1016/j.cca.2020.03.022 .

Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270 .

Lupia T, Scabini S, Mornese Pinna S, di Perri G, de Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: a new challenge. J Glob Antimicrob Resist. 2020;21:22–7. https://doi.org/10.1016/j.jgar.2020.02.021 .

Marasinghe, K.M., A systematic review investigating the effectiveness of face mask use in limiting the spread of COVID-19 among medically not diagnosed individuals: shedding light on current recommendations provided to individuals not medically diagnosed with COVID-19. Research Square. Preprint article. doi : https://doi.org/10.21203/rs.3.rs-16701/v1 . 2020 .

Mullins E, Evans D, Viner RM, O’Brien P, Morris E. Coronavirus in pregnancy and delivery: rapid review. Ultrasound Obstet Gynecol. 2020;55(5):586–92. https://doi.org/10.1002/uog.22014 .

Pang J, Wang MX, Ang IYH, Tan SHX, Lewis RF, Chen JIP, et al. Potential Rapid Diagnostics, Vaccine and Therapeutics for 2019 Novel coronavirus (2019-nCoV): a systematic review. J Clin Med. 2020;9(3):623.

Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, Villamizar-Peña R, Holguin-Rivera Y, Escalera-Antezana JP, et al. Clinical, laboratory and imaging features of COVID-19: a systematic review and meta-analysis. Travel Med Infect Dis. 2020;34:101623. https://doi.org/10.1016/j.tmaid.2020.101623 .

Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. https://doi.org/10.2214/AJR.20.23034 .

Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612–7. https://doi.org/10.1002/jmv.25735 .

Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis. 2020;94:91–5. https://doi.org/10.1016/j.ijid.2020.03.017 .

Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: challenges for fighting the storm. Eur J Clin Investig. 2020;50(3):e13209. https://doi.org/10.1111/eci.13209 .

Article   CAS   Google Scholar  

Hwang CS. Olfactory neuropathy in severe acute respiratory syndrome: report of a case. Acta Neurol Taiwanica. 2006;15(1):26–8.

Google Scholar  

Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, et al. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272–7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e .

Rajgor DD, Lee MH, Archuleta S, Bagdasarian N, Quek SC. The many estimates of the COVID-19 case fatality rate. Lancet Infect Dis. 2020;20(7):776–7. https://doi.org/10.1016/S1473-3099(20)30244-9 .

Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol. 2020;20(1):81. https://doi.org/10.1186/s12874-020-00972-6 .

Rombey T, Lochner V, Puljak L, Könsgen N, Mathes T, Pieper D. Epidemiology and reporting characteristics of non-Cochrane updates of systematic reviews: a cross-sectional study. Res Synth Methods. 2020;11(3):471–83. https://doi.org/10.1002/jrsm.1409 .

Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114–21. https://doi.org/10.1016/j.jclinepi.2019.08.010 .

Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13(1):224. https://doi.org/10.1186/s12916-015-0465-6 .

COVID-19 Rapid Reviews: Cochrane’s response so far. Available at: https://training.cochrane.org/resource/covid-19-rapid-reviews-cochrane-response-so-far . Accessed 1 June 2021.

Cochrane. Living systematic reviews. Available at: https://community.cochrane.org/review-production/production-resources/living-systematic-reviews . Accessed 1 June 2021.

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T. Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019;8(1):325. https://doi.org/10.1186/s13643-019-1248-5 .

Babic A, Poklepovic Pericic T, Pieper D, Puljak L. How to decide whether a systematic review is stable and not in need of updating: analysis of Cochrane reviews. Res Synth Methods. 2020;11(6):884–90. https://doi.org/10.1002/jrsm.1451 .

Lovato A, Rossettini G, de Filippis C. Sore throat in COVID-19: comment on “clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis”. J Med Virol. 2020;92(7):714–5. https://doi.org/10.1002/jmv.25815 .

Leung C. Comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1431–2. https://doi.org/10.1002/jmv.25912 .

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. Response to Char’s comment: comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1433. https://doi.org/10.1002/jmv.25924 .

Download references

Acknowledgments

We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from the Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Additional file 6 . We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.

This research received no external funding.

Author information

Authors and affiliations.

University Hospital and School of Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

Israel Júnior Borges do Nascimento & Milena Soriano Marcolino

Medical College of Wisconsin, Milwaukee, WI, USA

Israel Júnior Borges do Nascimento

Helene Fuld Health Trust National Institute for Evidence-based Practice in Nursing and Healthcare, College of Nursing, The Ohio State University, Columbus, OH, USA

Dónal P. O’Mathúna

School of Nursing, Psychotherapy and Community Health, Dublin City University, Dublin, Ireland

Department of Anesthesiology, Intensive Care and Pain Medicine, University of Münster, Münster, Germany

Thilo Caspar von Groote

Department of Sport and Health Science, Technische Universität München, Munich, Germany

Hebatullah Mohamed Abdulazeem

School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Callaghan, Australia

Ishanka Weerasekara

Department of Physiotherapy, Faculty of Allied Health Sciences, University of Peradeniya, Peradeniya, Sri Lanka

Cochrane Croatia, University of Split, School of Medicine, Split, Croatia

Ana Marusic, Irena Zakarija-Grkovic & Tina Poklepovic Pericic

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Cochrane Brazil, Evidence-Based Health Program, Universidade Federal de São Paulo, São Paulo, Brazil

Vinicius Tassoni Civile & Alvaro Nagib Atallah

Yorkville University, Fredericton, New Brunswick, Canada

Santino Filoso

Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada

Nicola Luigi Bragazzi

You can also search for this author in PubMed   Google Scholar

Contributions

IJBN conceived the research idea and worked as a project coordinator. DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP, ANA, SF, NLB and MSM were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. All authors revised the manuscript critically for the content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

Not required as data was based on published studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1..

Search strategies used in the study.

Additional file 2: Appendix 2.

Adjusted scoring of AMSTAR 2 used in this study for systematic reviews of studies that did not analyze interventions.

Additional file 3: Appendix 3.

List of excluded studies, with reasons.

Additional file 4: Appendix 4.

Table of overlapping studies, containing the list of primary studies included, their visual overlap in individual systematic reviews, and the number in how many reviews each primary study was included.

Additional file 5: Appendix 5.

A detailed explanation of AMSTAR scoring for each item in each review.

Additional file 6: Appendix 6.

List of members and affiliates of International Network of Coronavirus Disease 2019 (InterNetCOVID-19).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Borges do Nascimento, I.J., O’Mathúna, D.P., von Groote, T.C. et al. Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews. BMC Infect Dis 21 , 525 (2021). https://doi.org/10.1186/s12879-021-06214-4

Download citation

Received : 12 April 2020

Accepted : 19 May 2021

Published : 04 June 2021

DOI : https://doi.org/10.1186/s12879-021-06214-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus
• Evidence-based medicine
• Infectious diseases

BMC Infectious Diseases

ISSN: 1471-2334

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 04 February 2022

Studying the post-COVID-19 condition: research challenges, strategies, and importance of Core Outcome Set development

• Daniel Munblit   ORCID: orcid.org/0000-0001-9652-6856 1 , 2 , 3   na1 ,
• Timothy R. Nicholson 4   na1 ,
• Dale M. Needham 5 , 6 , 7   na1 ,
• Nina Seylanova 8   na1 ,
• Callum Parr 9 ,
• Jessica Chen 9 ,
• Alisa Kokorina 10 ,
• Louise Sigfrid 11 ,
• Danilo Buonsenso 12 , 13 , 14 ,
• Shinjini Bhatnagar 15 ,
• Ramachandran Thiruvengadam 15 ,
• Ann M. Parker 5 , 6 ,
• Jacobus Preller 16 ,
• Sergey Avdeev 17 ,
• Frederikus A. Klok 18 ,
• Allison Tong 19 ,
• Janet V. Diaz 20 ,
• Wouter De Groote 20 ,
• Nicoline Schiess 21 ,
• Athena Akrami 22 , 23 ,
• Frances Simpson 24 ,
• Piero Olliaro 25 ,
• Christian Apfelbacher 26 ,
• Regis Goulart Rosa 27 , 28 ,
• Jennifer R. Chevinsky 29 , 30 ,
• Sharon Saydah 29 , 31 ,
• Jochen Schmitt 32 ,
• Alla Guekht 3 , 10 ,
• Sarah L. Gorst 33 ,
• Jon Genuneit 34 ,
• Luis Felipe Reyes 35 , 36 ,
• Alan Asmanov 37 ,
• Margaret E. O’Hara 38 ,
• Janet T. Scott 39 ,
• Melina Michelen 25 , 40 ,
• Charitini Stavropoulou 40 ,
• John O. Warner 41 ,
• Margaret Herridge 42 , 43 &
• Paula R. Williamson 44   na1  

BMC Medicine volume  20 , Article number:  50 ( 2022 ) Cite this article

24k Accesses

68 Citations

52 Altmetric

Metrics details

A substantial portion of people with COVID-19 subsequently experience lasting symptoms including fatigue, shortness of breath, and neurological complaints such as cognitive dysfunction many months after acute infection. Emerging evidence suggests that this condition, commonly referred to as long COVID but also known as post-acute sequelae of SARS-CoV-2 infection (PASC) or post-COVID-19 condition , could become a significant global health burden.

While the number of studies investigating the post-COVID-19 condition is increasing, there is no agreement on how this new disease should be defined and diagnosed in clinical practice and what relevant outcomes to measure. There is an urgent need to optimise and standardise outcome measures for this important patient group both for clinical services and for research and to allow comparing and pooling of data.

Conclusions

A Core Outcome Set for post-COVID-19 condition should be developed in the shortest time frame possible, for improvement in data quality, harmonisation, and comparability between different geographical locations. We call for a global initiative, involving all relevant partners, including, but not limited to, healthcare professionals, researchers, methodologists, patients, and caregivers. We urge coordinated actions aiming to develop a Core Outcome Set (COS) for post-COVID-19 condition in both the adult and paediatric populations.

Peer Review reports

The coronavirus disease 2019 (COVID-19) pandemic has necessitated rapid responses from healthcare systems and research networks globally. Although a large amount of comprehensive data on acute symptoms and clinical management has been collected and analysed, there are currently no established clinical definition or Core Outcome Sets (COS). Indeed, even the terminology of the condition is debated with variable terms and definitions for the post-COVID-19 condition including long COVID , “long haulers”, post-acute sequelae of SARS-CoV-2 infection (PASC) , or post-COVID-19 condition , the term used by the World Health Organization (WHO). With over 217 million confirmed COVID-19 cases globally [ 1 ], post-COVID-19 condition risks affecting millions of people worldwide, making it an urgent research priority [ 2 ]. Although wide-scale vaccination may eventually lead to a decline in the number of COVID-19 cases, with cases rising worldwide, the pandemic is far from over. There is an urgent need for consensus on critically important core outcomes to be measured in post-COVID-19 condition . Establishing a COS will ensure that critically important outcomes are measured and reported in a consistent manner in research and practice settings. The consistent use of the most important outcomes across studies and clinical practice is essential to compare and collate the research findings with translation into clinical recommendations for patient care.

In this manuscript, we discuss the existing data known about post-COVID-19 condition research following WHO’s systematic approach to identifying research gaps [ 3 ] with the principal purpose of suggesting and outlining the implementation of a COS for the post-COVID-19 condition (i) to allow for the assessment of outcomes which are of the greatest relevance and importance to stakeholders and relevant target populations including patients, families, clinicians, researchers, health systems, public health policymakers, industry, and funding organisations; (ii) to improve the consistency and quality of data collection; and (iii) to build a foundation for data sharing for pooled analysis for meta-analysis and comparison of results across studies and global regions.

Given the current immediate need for an accepted post-COVID-19 condition COS, the development of a COS for the post-COVID-19 condition could act as a guide over the next few years until more is known and/or review or reassessment is indicated.

Post-COVID condition health consequences

Recent editorials [ 4 , 5 , 6 ] and National Institutes of Health (NIH) [ 7 ] and WHO [ 8 ] sponsored conferences have drawn attention to an increasing number of people experiencing health consequences following the acute phase of SARS-CoV-2 infection and are calling for research into the risk factors, clinical features, diagnosis, management, and outcomes. Increasing funding opportunities have subsequently followed [ 9 , 10 ]. It is important to note that most data regarding post-COVID-19 condition have been generated prior to the condition definition announcement. Thus, earlier studies may not fit the proposed definition criteria. Post-COVID-19 condition extends beyond the cardio-respiratory system to affect most other bodily systems both anatomically and physiologically [ 11 ]. Although causes of post-COVID-19 condition are unclear, persistent immune activation may be involved [ 12 ]. Risk factors for different syndromes of post-acute SARS-CoV-2 sequela have not been characterised, but it has been hypothesised that several post-COVID-19 condition phenotypes may exist, although pathophysiology, management, and outcomes are currently unknown.

Long-term health consequences of COVID-19 remain unknown, but reports suggest that prolonged symptom duration and limitations in functioning are common among hospitalised as well as non-hospitalised adults [ 13 , 14 ] and children [ 15 , 16 ]. The spectrum of long-lasting symptoms is wide and varies from mild discomfort to severe adverse effects on physical, cognitive, and psychosocial health [ 17 ], with important wider implications on functioning, including employment and school attendance.

Multiple studies from different countries found that many individuals experienced persistent symptoms 6 months after COVID-19, with fatigue or muscle weakness, sleep difficulties, and anxiety or depression among the most common sequelae [ 13 , 14 , 18 ]. A recent study suggests that although most COVID-19 survivors recover both physically and functionally a year after acute infection, some still experience problems with mobility, pain or discomfort, and anxiety or depression compared with non-COVID-19 controls [ 19 ]. The data emerging from the controlled studies are in agreement with the earlier reports. A recent analysis of the data from over 250,000 electronic health records demonstrated that more than one in three individuals had one or more features of post-COVID-19 condition recorded between 3 and 6 months after a diagnosis of COVID-19, which was significantly higher when compared with individuals with influenza [ 20 , 21 , 22 , 23 , 24 ]. Disease severity, female sex, and younger age were associated with a higher risk of post-COVID-19 condition development.

Yet, it is unknown whether persistent symptoms and associated abnormalities will fully resolve or whether some may leave life-long dysfunction. It is also worth noting that investigations into the post-COVID-19 condition can be difficult, with high loss to follow-up, frequent use of unvalidated measurement instruments, lack of inclusion of controls during the pandemic, and censoring of data (e.g. for death) not always fully considered in published studies. Differential diagnosis can be challenging with specific symptoms attributed to post-COVID-19 condition being a sign of an ongoing problem (e.g. dysautonomia in people reporting heart rate variability) [ 25 ].

Investigation of potential post-COVID-19 condition treatment options is still in its early days. Approaches mainly focus on rehabilitation and symptomatic management. Some experts suggest that antibodies and T cells able to recognise SARS-CoV-2 induced by vaccine “may help the immune system to stop the virus during its first few replications before it can establish hidden reservoirs in the body” [ 26 ]; however, the evidence regarding the effectiveness of SARS-CoV-2 vaccines in post-COVID-19 condition treatment is somehow conflicting [ 27 , 28 ]. One of the major obstacles in the development of intervention strategies for post-COVID-19 condition is the lack of agreed outcomes to be assessed in clinical trials.

The pandemic and subsequent mitigation strategies have also had a substantial impact on the psychosocial well-being of the general population worldwide, with many people experiencing anxiety and depression, due to isolation, economic instability, job insecurity, sickness/death of infected family members, COVID-19-related stigma, lack of trust in government agencies, and constant media attention focused on the pandemic threats [ 29 ]. Disruption of care for those with pre-existing conditions has also had a large impact. For example, according to the WHO Pulse survey on continuity of essential health services during the COVID-19 pandemic [ 30 ], 45% of countries still reported disruptions to services for mental, neurological, and substance use disorders in the first quarter of 2021. On a similar scale, rehabilitation services disruptions continue to be reported by 53% (of 89 countries).

In addition, indirect impacts of COVID-19 on mental health [ 31 ], psychosocial, and neurological sequelae have been reported in adults following COVID-19 [ 32 ], and many patients are facing a variety of consequences including fatigue, shortness of breath, and cognitive dysfunction as well as reduced quality of life [ 18 ] with an impact on everyday functioning, even months following acute infection [ 19 ].

With millions of people affected by COVID-19, even a small percentage developing the post-COVID-19 condition will result in a detrimental effect on society and public health, with many people in need of long-term follow-up, management, and support[ 5 ]. A recent study has reported that previously hospitalised patients with COVID-19 had increased rates of multiorgan dysfunction compared with the general population [ 33 ].

Terminology and clinical definitions

With many unresolved issues regarding this condition, the inclusion of patients’ perspectives has become increasingly important to the development of a COS. Importantly, there is currently no agreement on a clinical definition and which outcomes should be measured and how they should be measured. WHO has recently completed a Delphi consensus to finalise a clinical case definition of post-COVID-19 condition as described below [ 34 ]. This official WHO definition was published recently and is formulated as the following: “post-COVID-19 condition occurs in individuals with a history of probable or confirmed SARS CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms and that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, and cognitive dysfunction, but also others and generally have an impact on everyday functioning. Symptoms may be new onset following the initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms may also fluctuate or relapse over time” [ 35 ].

Other organisations have also proposed interim definitions such as the United Kingdom National Institute for Health and Care Excellence (NICE) who suggest an interim definition of post-COVID-19 syndrome as “signs and symptoms that develop during or after an infection consistent with COVID-19, continue for more than 12 weeks, and are not explained by alternative diagnoses” [ 36 ]. Due to the scarcity and high degree of heterogeneity of existing studies, the definition may change with the emergence of new evidence, but diversity in diagnostic criteria, methodology, and outcomes measured may slow down the progress. There is a need for terminology harmonisation, and consensus between major public health and government research organisations and professional bodies should be reached for the benefit and convenience of clinicians, researchers, and most importantly patients. With the growing evidence on COVID-19 consequences, there is a risk that the number of different terms used for the post-COVID-19 condition will increase further with increasing phenotyping of this condition.

Core Outcome Set definition and relevance

Outcomes are measured in clinical research or practice to help evaluate relevant associations, safety and efficacy, risk factors, and effectiveness of interventions [ 37 ]. The lack of agreed upon outcomes and associated measurement instruments may complicate the evidence synthesis due to the inability to pool data in meta-analyses [ 38 ], resulting in a heterogeneous, incomplete, and low-quality evidence base and a barrier to clinical guideline development and policymaking. A COS comprises a minimum list of outcomes recommended for assessment in all studies, clinical practice setting, or both, for a specific condition and is necessary to harmonise research outputs and increase their comparability, quality, and generalisability to ensure their relevance to all interested partners. The failure to consider the views of patients and their families in selecting outcomes may result in less relevant outcome measures being evaluated, while the most important outcomes may be missed.

Core Outcome Sets for COVID-19 and post-COVID-19 condition

A number of COVID-19-related COS are registered at the Core Outcome Measures in Effectiveness Trials (COMET) Initiative web-registry of planned, ongoing, and completed COS studies [ 39 ]. These initiatives provide clinicians, researchers, and policymakers with important information on the relevant aspects of COVID-19 and allow for the generation of harmonised high-quality data. Notably, COVID-19 COS development projects were undertaken in a rapid fashion, in contrast with the usual COS development process which typically takes several years.

Although none of the available COVID-19-related COS projects is solely dedicated to the post-COVID-19 condition , some include outcomes for “rehabilitation period”, “longer term impacts”, and “recovery” outcomes [ 40 , 41 ]. Despite a well-established and standardised approach to COS development, which is generally guided by the COMET Handbook, the development of COS for the post-COVID-19 condition may be a complex task given the diversity and multisystem nature of infection sequelae [ 33 ]. COS previously developed for other conditions, which may be relevant for people recovering from COVID-19, should be considered a potential option for certain groups of individuals (e.g. acute respiratory failure/acute respiratory distress syndrome survivors after hospital discharge) [ 42 ]. We reviewed studies; assessed in the living systematic review of long COVID [ 43 ], data from the clinical trial registries, and available case report forms; and outlined outcomes previously measured in long COVID studies (Table 1 ).

Considerations and limitations within vulnerable populations

The development of a COS in vulnerable populations such as people with disabilities/comorbidities, undergoing complex treatment (e.g. chemotherapy or transplantation), elderly, racial and ethnic minority groups, pregnant women, and children have special challenges that merit additional considerations. These challenges revolve not just around the constellation of symptoms but also the right time for the assessment and the individual on whom these are assessed. For a pregnant woman who was infected in the first trimester and recovered fully before delivery, any adverse birth outcome, such as preterm birth, stillbirth, or pregnancy complication (e.g. preeclampsia or gestational diabetes), may or may not be part of the post-COVID-19 condition [ 44 ]. Whether the congenital anomalies in the neonate or any neonatal complication following maternal COVID-19 can be defined as a potential post-COVID-19 condition for the neonate or infant requires consideration. It is particularly important to differentiate an adverse birth outcome that could be attributed to maternal COVID-19 from one that would have occurred otherwise due to other risk factors, irrespective of maternal COVID-19 status. This is similar to ruling out other aetiologies within the general population (non-pregnant adults).

Considerations and limitations within low-middle income settings

There are multiple implications to the COVID-19 pandemic on low- and middle-income countries (LMICs) including lack of available healthcare resources to meet the needs of the local population, not only with COVID-19 infection, but for other acute and chronic conditions as well [ 45 ]. Another anticipated challenge for LMICs is the recognition of the post-COVID-19 condition . Post-acute care of physical, cognitive, and mental health disabilities may go under-recognised, especially in low-resource contexts in which all efforts are focused on containing COVID-19 dissemination and providing an appropriate care for severely ill patients. This may impact research on the post-COVID-19 condition as well as identification and management in LMICs.

Current definitions for the assessment of specific outcomes involve advanced laboratory and imaging techniques which require resources and skills. Such skills and resources may not be readily available in resource-limited settings. An inclusive approach should be taken while compiling the COS, by including alternate definitions and methods of measurement which may be acceptable for low-middle income settings. Extraordinary care must be taken to strike a balance between accuracy of the assessment and feasibility across the globe. The development of a COS for post-COVID-19 condition should account for cultural and social differences and restrictions in access and resources. Clinicians, researchers, and patient representatives from LMICs should be engaged in the COS development process to ensure global representability and future applicability of the COS.

Limitations to existing research

Although attention to the problem of the post-COVID-19 condition is increasing, there are still many unanswered questions and important limitations impacting research quality and understanding of COVID-19 sequelae.

We outline these as well as potential mitigation strategies following a systematic approach [ 3 ] or defining research priorities through planning, implementation, publishing, and evaluation phases in Table 2 . The table was drafted by DM and critically appraised and approved by all the authors. Post-COVID-19 condition still has no consensus definition, well-defined clinical phenotypes, or clearly explained underlying physiological mechanisms. WHO has highlighted “three Rs” related to post-COVID-19 condition —recognition, research, and rehabilitation, and initiated working groups aiming to provide a post-COVID-19 condition clinical definition [ 46 ] and outline plausible explanations of the physiological mechanisms as well as proposed an interim clinical case definition through a multi-disciplinary, gender-based, international Delphi consensus [ 34 ].

The number of studies assessing post-COVID-19 condition is increasing, generating a large amount of data, where validity remains unknown. There is a large variability in reporting and quantification of post-COVID-19 condition symptoms among the studies. It is important to note that not only symptom presence is essential, but symptom duration and severity also merit consideration. The lack of pre-morbid data for comparison is one of the major limitations of post-COVID-19 condition research. Any abnormalities found are normally attributed to post-COVID-19 condition assuming that the patient did not have asymptomatic abnormal testing before infection.

Few international initiatives have created instruments for data collection. The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) has developed follow-up protocols and surveys for adults and children to assess the prevalence and risk factors for long-term physical and psychosocial health consequences following COVID-19 diagnosis. A post-COVID-19 condition case report form has been designed by the WHO to report standardised clinical data from individuals after hospital discharge or after the acute illness to examine the medium- and long-term consequences of COVID-19. Although these instruments assist harmonised data collection, an increasing number of tools from reputable organisations may result in data heterogeneity with different centres prioritising different instruments.

There are few ongoing initiatives tackling the problem of data heterogeneity by systematically reviewing available evidence in the live format [ 43 ], which may inform COS initiatives and assist with the long list of outcome development. However, systematic reviews will not address the problem of instrument validity. Assessment of the validity may take a long time, and meanwhile, a COS should be developed.

A significant gap and limitation within post-COVID-19 condition research exist within paediatric and adolescent development considering that life-long consequences may exist [ 47 ]. Outcomes of interest in children and adolescents may be very different to the adult population, and COS for this age group should be specifically developed engaging the children and adolescents themselves, as well as their parents and carers.

This manuscript was written by a multidisciplinary (allergists, critical care specialists, ENT specialists, infectionists, immunologists, neurologists, psychiatrists, paediatricians, pulmonologists, specialists in global and public health experts, epidemiologists, methodologists, rehabilitation specialists, and people with lived experience of post-COVID-19 condition ), gender-balanced, international group of experts, including members of the ISARIC Consortium, US Centers for Disease Control and Prevention (CDC), experts involved in the WHO post-COVID condition clinical characterisation group, leads of international COVID-19 cohorts, members of Core Outcome Measures for post-COVID-19 condition/long COVID initiative and patient representatives, to outline the unmet needs and justification for Core Outcome Set development for the post-COVID-19 condition which may become a major public health burden. Previous research in various medical fields has demonstrated the importance and usefulness of COS in both research and clinical practice. There is a need to rapidly develop a COS for the post-COVID-19 condition which will allow for the improvement in data quality, harmonisation, and comparability between different geographical locations. The joint initiative requires input from all relevant partners, including, but not limited to, healthcare professionals, researchers, methodologists, patients, and carers. We urge local and international funding agencies to provide support for coordinated actions aiming to develop COS for post-COVID-19 condition in adults and children.

Availability of data and materials

Not applicable.

Abbreviations

US Centers for Disease Control and Prevention

Core Outcome Measures in Effectiveness Trials Initiative

• Core Outcome Set

Coronavirus disease 2019

International Severe Acute Respiratory and Emerging Infection Consortium

Low- and middle-income countries

National Institute for Health and Care Excellence

National Institutes of Health

• Post-acute sequelae of SARS-CoV-2 infection

Severe acute respiratory syndrome-related coronavirus 2

World Health Organization

World Health Organization. WHO Coronavirus (COVID-19) Dashboard [ https://covid19.who.int/ ]

Lerner AM, Robinson DA, Yang L, Williams CF, Newman LM, Breen JJ, et al. Toward understanding COVID-19 recovery: National Institutes of Health Workshop on Postacute COVID-19. Ann Intern Med. 2021;174(7):999–1003. https://doi.org/10.7326/M21-1043 .

Article   PubMed   Google Scholar  

World Health Organization. A systematic approach for undertaking a research priority setting exercise [ https://apps.who.int/iris/bitstream/handle/10665/334408/9789240009622-eng.pdf ]

The Lancet. Facing up to long COVID. Lancet. 2020;396(10266):1861. https://doi.org/10.1016/S0140-6736(20)32662-3 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Meeting the challenge of long COVID. Nat Med 2020, 26(12):1803.

The Lancet. Understanding long COVID: a modern medical challenge. Lancet. 2021;398(10302):725. https://doi.org/10.1016/S0140-6736(21)01900-0 .

National Institutes of Health. NIH experts discuss post-acute COVID-19 [ https://www.nih.gov/news-events/news-releases/nih-experts-discuss-post-acute-covid-19 ]

National Institutes of Health. Expanding our understanding of post COVID-19 condition: report of a WHO webinar. 2021 [ https://www.who.int/publications/i/item/9789240025035 ]

National Institutes of Health. NIH makes first infrastructure awards to support research on post COVID conditions [ https://www.nih.gov/about-nih/who-we-are/nih-director/statements/nih-makes-first-infrastructure-awards-support-research-post-covid-conditions ]

National Institute for Health Research. £19.6 million awarded to new research studies to help diagnose and treat long COVID [ https://www.nihr.ac.uk/news/196-million-awarded-to-new-research-studies-to-help-diagnose-and-treat-long-covid/28205 ]

Chevinsky JR, Tao G, Lavery AM, Kukielka EA, Click ES, Malec D, et al. Late conditions diagnosed 1-4 months following an initial COVID-19 encounter: a matched cohort study using inpatient and outpatient administrative data - United States, March 1-June 30, 2020. Clin Infect Dis. 2021;73(Supplement_1):S16. https://doi.org/10.1093/cid/ciab338 .

Article   CAS   Google Scholar  

Peluso MJ, Lu S, Tang AF, Durstenfeld MS, Ho HE, Goldberg SA, Forman CA, Munter SE, Hoh R, Tai V, Chenna A, Yee BC, Winslow JW, Petropoulos CJ, Greenhouse B, Hunt PW, Hsue PY, Martin JN, Daniel Kelly J, Glidden DV, Deeks SG, Henrich TJ. Markers of Immune Activation and Inflammation in Individuals With Postacute Sequelae of Severe Acute Respiratory Syndrome Coronavirus 2 Infection. J Infect Dis. 2021;224(11):1839–48.

Huang C, Huang L, Wang Y, Li X, Ren L, Gu X, et al. 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study. Lancet. 2021;397(10270):220–32. https://doi.org/10.1016/S0140-6736(20)32656-8 .

Munblit D, Bobkova P, Spiridonova E, Shikhaleva A, Gamirova A, Blyuss O, et al. Incidence and risk factors for persistent symptoms in adults previously hospitalised for COVID-19. Clin Exp Allergy. 2021;51(9):1107–20. https://doi.org/10.1111/cea.13997 .

Buonsenso D, Munblit D, De Rose C, Sinatti D, Ricchiuto A, Carfi A, et al. Preliminary evidence on long COVID in children. Acta Paediatr. 2021;110(7):2208–11. https://doi.org/10.1111/apa.15870 .

Osmanov IM, Spiridonova E, Bobkova P, Gamirova A, Shikhaleva A, Andreeva M, et al. Risk factors for long covid in previously hospitalised children using the ISARIC Global follow-up protocol: a prospective cohort study. Eur Respir J. 2021:2101341. https://doi.org/10.1183/13993003.01341-2021 .

Crispo A, Bimonte S, Porciello G, Forte CA, Cuomo G, Montagnese C, et al. Strategies to evaluate outcomes in long-COVID-19 and post-COVID survivors. Infect Agent Cancer. 2021;16(1):62. https://doi.org/10.1186/s13027-021-00401-3 .

Sigfrid L, Drake TM, Pauley E, Jesudason EC, Olliaro P, Lim WS, et al. Long COVID in adults discharged from UK hospitals after Covid-19: a prospective, multicentre cohort study using the ISARIC WHO Clinical Characterisation Protocol. Lancet Reg Health Eur. 2021;8:100186. https://doi.org/10.1016/j.lanepe.2021.100186 .

Article   PubMed   PubMed Central   Google Scholar  

Huang L, Yao Q, Gu X, Wang Q, Ren L, Wang Y, et al. 1-year outcomes in hospital survivors with COVID-19: a longitudinal cohort study. Lancet. 2021;398(10302):747–58. https://doi.org/10.1016/S0140-6736(21)01755-4 .

Mendez R, Latorre A, Gonzalez-Jimenez P, Feced L, Bouzas L, Yepez K, et al. Reduced diffusion capacity in COVID-19 survivors. Ann Am Thorac Soc. 2021;18(7):1253–5. https://doi.org/10.1513/AnnalsATS.202011-1452RL .

Townsend L, Dowds J, O’Brien K, Sheill G, Dyer AH, O’Kelly B, et al. Persistent poor health post-COVID-19 is not associated with respiratory complications or initial disease severity. Ann Am Thorac Soc. 2021;18(6):997–1003. https://doi.org/10.1513/AnnalsATS.202009-1175OC .

Sudre CH, Murray B, Varsavsky T, Graham MS, Penfold RS, Bowyer RC, et al. Attributes and predictors of long COVID. Nat Med. 2021;27(4):626–31. https://doi.org/10.1038/s41591-021-01292-y .

Raman B, Cassar MP, Tunnicliffe EM, Filippini N, Griffanti L, Alfaro-Almagro F, et al. Medium-term effects of SARS-CoV-2 infection on multiple vital organs, exercise capacity, cognition, quality of life and mental health, post-hospital discharge. EClinicalMedicine. 2021;31:100683. https://doi.org/10.1016/j.eclinm.2020.100683 .

Taquet M, Dercon Q, Luciano S, Geddes JR, Husain M, Harrison PJ. Incidence, co-occurrence, and evolution of long-COVID features: a 6-month retrospective cohort study of 273,618 survivors of COVID-19. PLoS Med. 2021;18(9):e1003773. https://doi.org/10.1371/journal.pmed.1003773 .

Barizien N, Le Guen M, Russel S, Touche P, Huang F, Vallee A. Clinical characterization of dysautonomia in long COVID-19 patients. Sci Rep. 2021;11(1):14042. https://doi.org/10.1038/s41598-021-93546-5 .

Ledford H. Do vaccines protect against long COVID? What the data say. Nature. 2021;599(7886):546–8. https://doi.org/10.1038/d41586-021-03495-2 .

Article   CAS   PubMed   Google Scholar  

Arnold DT, Milne A, Samms E, Stadon L, Maskell NA, Hamilton FW. Symptoms after COVID-19 vaccination in patients with persistent symptoms after acute infection: a case series. Ann Intern Med. 2021;174(9):1334–6. https://doi.org/10.7326/M21-1976 .

Office for National Statistics. Coronavirus (COVID-19) vaccination and self-reported long COVID in the UK: 2021 [ https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/articles/coronaviruscovid19vaccinationandselfreportedlongcovidintheuk/25october2021 ]

Nekliudov NA, Blyuss O, Cheung KY, Petrou L, Genuneit J, Sushentsev N, et al. Excessive media consumption about COVID-19 is associated with increased state anxiety: outcomes of a large online survey in Russia. J Med Internet Res. 2020;22(9):e20955. https://doi.org/10.2196/20955 .

World Health Organization. Second round of the national pulse survey on continuity of essential health services during the COVID-19 pandemic [ https://www.who.int/publications/i/item/WHO-2019-nCoV-EHS-continuity-survey-2021.1 ]

De Picker LJ, Dias MC, Benros ME, Vai B, Branchi I, Benedetti F, et al. Severe mental illness and European COVID-19 vaccination strategies. Lancet Psychiatry. 2021;8(5):356–9. https://doi.org/10.1016/S2215-0366(21)00046-8 .

Williams S, Wynford-Thomas R, Robertson NP. Long-COVID: neurological manifestations and management. J Neurol. 2021;268(12):4915–7. https://doi.org/10.1007/s00415-021-10847-5 .

Ayoubkhani D, Khunti K, Nafilyan V, Maddox T, Humberstone B, Diamond I, et al. Post-covid syndrome in individuals admitted to hospital with covid-19: retrospective cohort study. BMJ. 2021;372:n693. https://doi.org/10.1136/bmj.n693 .

Soriano JB, Murthy S, Marshall JC, Relan P, Diaz JV. WHO Clinical Case Definition Working Group on Post-COVID-19 Condition. A clinical case definition of post-COVID-19 condition by a Delphi consensus. Lancet Infect Dis. 2021:S1473-3099(21)00703–9.

World Health Organization. A clinical case definition of post COVID-19 condition by a Delphi consensus, 2021 [ https://www.who.int/publications/i/item/WHO-2019-nCoV-Post_COVID-19_condition-Clinical_case_definition-2021.1 ]

The National Institute for Health and Care Excellence. COVID-19 rapid guideline: managing the long-term effects of COVID-19 [ https://www.nice.org.uk/guidance/indevelopment/gid-ng10179 ]

Schmitt J, Apfelbacher C, Spuls PI, Thomas KS, Simpson EL, Furue M, et al. The Harmonizing Outcome Measures for Eczema (HOME) roadmap: a methodological framework to develop core sets of outcome measurements in dermatology. J Invest Dermatol. 2015;135(1):24–30. https://doi.org/10.1038/jid.2014.320 .

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. https://doi.org/10.1136/bmj.327.7414.557 .

Core Outcome Set developers’ response to COVID-19. 2021. [ https://comet-initiative.org/Studies/Details/1538 ]

Tong A, Baumgart A, Evangelidis N, Viecelli AK, Carter SA, Azevedo LC, et al. Core outcome measures for trials in people with coronavirus disease 2019: respiratory failure, multiorgan failure, shortness of breath, and recovery. Crit Care Med. 2021;49(3):503–16. https://doi.org/10.1097/CCM.0000000000004817 .

Tong A, Elliott JH, Azevedo LC, Baumgart A, Bersten A, Cervantes L, et al. Core Outcomes Set for trials in people with coronavirus disease 2019. Crit Care Med. 2020;48(11):1622–35. https://doi.org/10.1097/CCM.0000000000004585 .

Needham DM, Sepulveda KA, Dinglas VD, Chessare CM, Friedman LA, Bingham CO 3rd, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An International Modified Delphi Consensus Study. Am J Respir Crit Care Med. 2017;196(9):1122–30. https://doi.org/10.1164/rccm.201702-0372OC .

Michelen M, Manoharan L, Elkheir N, Cheng V, Dagens A, Hastie C, O'Hara M, Suett J, Dahmash D, Bugaeva P, Rigby I, Munblit D, Harriss E, Burls A, Foote C, Scott J, Carson G, Olliaro P, Sigfrid L, Stavropoulou C. Characterising long COVID: a living systematic review. BMJ Glob Health. 2021;6(9):e005427.

Villar J, Ariff S, Gunier RB, Thiruvengadam R, Rauch S, Kholin A, et al. Maternal and neonatal morbidity and mortality among pregnant women with and without COVID-19 infection: the INTERCOVID Multinational Cohort Study. JAMA Pediatr. 2021;175(8):817–26. https://doi.org/10.1001/jamapediatrics.2021.1050 .

Bong CL, Brasher C, Chikumba E, McDougall R, Mellin-Olsen J, Enright A. The COVID-19 pandemic: effects on low- and middle-income countries. Anesth Analg. 2020;131(1):86–92. https://doi.org/10.1213/ANE.0000000000004846 .

Wise J. Long COVID: WHO calls on countries to offer patients more rehabilitation. BMJ. 2021;372:n405. https://doi.org/10.1136/bmj.n405 .

Munblit D, Sigfrid L, Warner JO. Setting priorities to address research gaps in long-term COVID-19 outcomes in children. JAMA Pediatr. 2021;175(11):1095–6. https://doi.org/10.1001/jamapediatrics.2021.2281 .

Download references

Acknowledgements

We would like to thank Nikita A Nekliudov for his help with the literature search for this manuscript.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the International Severe Acute Respiratory and Emerging Infection Consortium, and the World Health Organisation.

This manuscript received no specific funding.

Author information

Daniel Munblit, Timothy Nicholson, Dale M. Needham, Nina Seylanova and Paula R. Williamson contributed equally to this work.

Authors and Affiliations

Department of Paediatrics and Paediatric Infectious Diseases, Institute of Child’s Health, Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia

Daniel Munblit

Inflammation, Repair and Development Section, National Heart and Lung Institute, Faculty of Medicine, Imperial College London, London, UK

Research and Clinical Center for Neuropsychiatry, Moscow, Russia

Daniel Munblit & Alla Guekht

Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK

Timothy R. Nicholson

Outcomes After Critical Illness and Surgery (OACIS) Research Group, Johns Hopkins University, Baltimore, MD, USA

Dale M. Needham & Ann M. Parker

Pulmonary and Critical Care Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA

Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, USA

Dale M. Needham

Sechenov Biomedical Science and Technology Park, Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia

Nina Seylanova

Faculty of Medicine, Imperial College London, London, UK

Callum Parr & Jessica Chen

Pirogov Russian National Research Medical University, Moscow, Russia

Alisa Kokorina & Alla Guekht

ISARIC Global Support Centre, Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK

Louise Sigfrid

Department of Woman and Child Health and Public Health, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy

Danilo Buonsenso

Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy

Global Health Research Institute, Istituto di Igiene, Università Cattolica del Sacro Cuore, Roma, Italy

Maternal and Child Health Program, Translational Health Science and Technology Institute, Faridabad, Delhi, National Capital Region, India

Shinjini Bhatnagar & Ramachandran Thiruvengadam

Clinical Management, WHO, WHE, Geneva, Switzerland

Jacobus Preller

Department of Pulmonology, Sechenov First Moscow State Medical University (Sechenov University), Moscow, Russia

Sergey Avdeev

Department of Medicine - Thrombosis and Hemostasis, Leiden University Medical Center, Leiden, the Netherlands

Frederikus A. Klok

Sydney School of Public Health, The University of Sydney, Sydney, Australia

Allison Tong

NCD Department, Rehabilitation Programme, WHO, Geneva, Switzerland

Janet V. Diaz & Wouter De Groote

WHO Brain Health Unit, Geneva, Switzerland

Nicoline Schiess

Sainsbury Wellcome Centre, UCL, London, UK

Athena Akrami

Patient-Led Research Collaborative, Washington, DC, USA

Coventry University, Coventry, UK

Frances Simpson

ISARIC Global Support Centre, Nuffield Department of Medicine, University of Oxford, Oxford, UK

Piero Olliaro & Melina Michelen

Institute of Social Medicine and Health Systems Research, Faculty of Medicine, Otto von Guericke University Magdeburg, Magdeburg, Germany

Christian Apfelbacher

Critical Care Department, Hospital Moinhos de Vento, Porto Alegre, Brazil

Regis Goulart Rosa

Brazilian Research in Intensive Care Network (BRICNet), São Paulo, Brazil

COVID-19 Response Team, Centers for Disease Control and Prevention, Atlanta, GA, USA

Jennifer R. Chevinsky & Sharon Saydah

Epidemic Intelligence Service, Centers for Disease Control and Prevention, Atlanta, GA, USA

Jennifer R. Chevinsky

Respiratory Viruses Branch, Division of Viral Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA

Sharon Saydah

Center for Evidence-Based Healthcare, Medical Faculty Carl Gustav Carus, TU Dresden, Dresden, Germany

Jochen Schmitt

Department of Health Data Science, University of Liverpool, Liverpool, UK

Sarah L. Gorst

Paediatric Epidemiology, Department of Pediatrics, Medical Faculty, Leipzig University, Leipzig, Germany

Jon Genuneit

Universidad de La Sabana, Chía, Colombia

Luis Felipe Reyes

Clínica Universidad de La Sabana, Chía, Colombia

The Research and Clinical Institute for Pediatrics named after Academician Yuri Veltischev of the Pirogov Russian National Research Medical University, Moscow, Russia

Alan Asmanov

Long COVID Support, London, UK

Margaret E. O’Hara

MRC-University of Glasgow, Centre for Virus Research, Glasgow, UK

Janet T. Scott

School of Health Sciences, City, University of London, London, UK

Melina Michelen & Charitini Stavropoulou

Paediatric Infectious Diseases, Imperial College Healthcare NHS Trust, London, UK

John O. Warner

Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, ON, Canada

Margaret Herridge

Department of Medicine, University Health Network, Toronto, ON, Canada

MRC/NIHR Trials Methodology Research Partnership, Department of Health Data Science, University of Liverpool (a member of Liverpool Health Partners), Liverpool, UK

Paula R. Williamson

You can also search for this author in PubMed   Google Scholar

Contributions

DM drafted the manuscript. All authors critically appraised and edited the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Daniel Munblit .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

DM, TRN, DN, and PW are leading the development of the Core Outcome Set for post-COVID-19 condition as a part of the PC-COS project team. The other authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Munblit, D., Nicholson, T., Needham, D.M. et al. Studying the post-COVID-19 condition: research challenges, strategies, and importance of Core Outcome Set development. BMC Med 20 , 50 (2022). https://doi.org/10.1186/s12916-021-02222-y

Download citation

Received : 19 October 2021

Accepted : 20 December 2021

Published : 04 February 2022

DOI : https://doi.org/10.1186/s12916-021-02222-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• COVID-19 sequalae
• Post-COVID-19 condition

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine (2021)

Chapter: 8 major findings and research questions, 8 major findings and research questions, introduction.

The COVID-19 pandemic, which began in late 2019, created unprecedented global disruption and infused a significant level of uncertainty into the lives of individuals, both personally and professionally, around the world throughout 2020. The significant effect on vulnerable populations, such as essential workers and the elderly, is well documented, as is the devastating effect the COVID-19 pandemic had on the economy, particularly brick-and-mortar retail and hospitality and food services. Concurrently, the deaths of unarmed Black people at the hands of law enforcement officers created a heightened awareness of the persistence of structural injustices in U.S. society.

Against the backdrop of this public health crisis, economic upheaval, and amplified social consciousness, an ad hoc committee was appointed to review the potential effects of the COVID-19 pandemic on women in academic science, technology, engineering, mathematics, and medicine (STEMM) during 2020. The committee’s work built on the National Academies of Sciences, Engineering, and Medicine report Promising Practices for Addressing the Underrepresentation of Women in Science, Engineering, and Medicine: Opening Doors (the Promising Practices report), which presents evidence-based recommendations to address the well-established structural barriers that impede the advancement of women in STEMM. However, the committee recognized that none of the actions identified in the Promising Practices report were conceived within the context of a pandemic, an economic downturn, or the emergence of national protests against structural racism. The representation and vitality of academic women in STEMM had already warranted national attention prior to these events, and the COVID-19

pandemic appeared to represent an additional risk to the fragile progress that women had made in some STEMM disciplines. Furthermore, the future will almost certainly hold additional, unforeseen disruptions, which underscores the importance of the committee’s work.

In times of stress, there is a risk that the divide will deepen between those who already have advantages and those who do not. In academia, senior and tenured academics are more likely to have an established reputation, a stable salary commitment, and power within the academic system. They are more likely, before the COVID-19 pandemic began, to have established professional networks, generated data that can be used to write papers, and achieved financial and job security. While those who have these advantages may benefit from a level of stability relative to others during stressful times, those who were previously systemically disadvantaged are more likely to experience additional strain and instability.

As this report has documented, during 2020 the COVID-19 pandemic had overall negative effects on women in academic STEMM in areas such productivity, boundary setting and boundary control, networking and community building, burnout rates, and mental well-being. The excessive expectations of caregiving that often fall on the shoulders of women cut across career timeline and rank (e.g., graduate student, postdoctoral scholar, non-tenure-track and other contingent faculty, tenure-track faculty), institution type, and scientific discipline. Although there have been opportunities for innovation and some potential shifts in expectations, increased caregiving demands associated with the COVID-19 pandemic in 2020, such as remote working, school closures, and childcare and eldercare, had disproportionately negative outcomes for women.

The effects of the COVID-19 pandemic on women in STEMM during 2020 are understood better through an intentionally intersectional lens. Productivity, career, boundary setting, mental well-being, and health are all influenced by the ways in which social identities are defined and cultivated within social and power structures. Race and ethnicity, sexual orientation, gender identity, academic career stage, appointment type, institution type, age, and disability status, among many other factors, can amplify or diminish the effects of the COVID-19 pandemic for a given person. For example, non-cisgender women may be forced to return to home environments where their gender identity is not accepted, increasing their stress and isolation, and decreasing their well-being. Women of Color had a higher likelihood of facing a COVID-19–related death in their family compared with their white, non-Hispanic colleagues. The full extent of the effects of the COVID-19 pandemic for women of various social identities was not fully understood at the end of 2020.

Considering the relative paucity of women in many STEMM fields prior to the COVID-19 pandemic, women are more likely to experience academic isolation, including limited access to mentors, sponsors, and role models that share gender, racial, or ethnic identities. Combining this reality with the physical isolation stipulated by public health responses to the COVID-19 pandemic,

women in STEMM were subject to increasing isolation within their fields, networks, and communities. Explicit attention to the early indicators of how the COVID-19 pandemic affected women in academic STEMM careers during 2020, as well as attention to crisis responses throughout history, may provide opportunities to mitigate some of the long-term effects and potentially develop a more resilient and equitable academic STEMM system.

MAJOR FINDINGS

Given the ongoing nature of the COVID-19 pandemic, it was not possible to fully understand the entirety of the short- or long-term implications of this global disruption on the careers of women in academic STEMM. Having gathered preliminary data and evidence available in 2020, the committee found that significant changes to women’s work-life boundaries and divisions of labor, careers, productivity, advancement, mentoring and networking relationships, and mental health and well-being have been observed. The following findings represent those aspects that the committee agreed have been substantiated by the preliminary data, evidence, and information gathered by the end of 2020. They are presented either as Established Research and Experiences from Previous Events or Impacts of the COVID-19 Pandemic during 2020 that parallel the topics as presented in the report.

Established Research and Experiences from Previous Events

___________________

1 This finding is primarily based on research on cisgender women and men.

Impacts of the COVID-19 Pandemic during 2020

Research questions.

While this report compiled much of the research, data, and evidence available in 2020 on the effects of the COVID-19 pandemic, future research is still needed to understand all the potential effects, especially any long-term implications. The research questions represent areas the committee identified for future research, rather than specific recommendations. They are presented in six categories that parallel the chapters of the report: Cross-Cutting Themes; Academic Productivity and Institutional Responses; Work-Life Boundaries and Gendered Divisions of Labor; Collaboration, Networking, and Professional Societies; Academic Leadership and Decision-Making; and Mental Health and Well-being. The committee hopes the report will be used as a basis for continued understanding of the impact of the COVID-19 pandemic in its entirety and as a reference for mitigating impacts of future disruptions that affect women in academic STEMM. The committee also hopes that these research questions may enable academic STEMM to emerge from the pandemic era a stronger, more equitable place for women. Therefore, the committee identifies two types of research questions in each category; listed first are those questions aimed at understanding the impacts of the disruptions from the COVID-19 pandemic, followed by those questions exploring the opportunities to help support the full participation of women in the future.

Cross-Cutting Themes

• What are the short- and long-term effects of the COVID-19 pandemic on the career trajectories, job stability, and leadership roles of women, particularly of Black women and other Women of Color? How do these effects vary across institutional characteristics, 2 discipline, and career stage?

2 Institutional characteristics include different institutional types (e.g., research university, liberal arts college, community college), locales (e.g., urban, rural), missions (e.g., Historically Black Colleges and Universities, Hispanic-Serving Institutions, Asian American/Native American/Pacific Islander-Serving Institutions, Tribal Colleges and Universities), and levels of resources.

• How did the confluence of structural racism, economic hardships, and environmental disruptions affect Women of Color during the COVID-19 pandemic? Specifically, how did the murder of George Floyd, Breonna Taylor, and other Black citizens impact Black women academics’ safety, ability to be productive, and mental health?
• How has the inclusion of women in leadership and other roles in the academy influenced the ability of institutions to respond to the confluence of major social crises during the COVID-19 pandemic?
• How can institutions build on the involvement women had across STEMM disciplines during the COVID-19 pandemic to increase the participation of women in STEMM and/or elevate and support women in their current STEMM-related positions?
• How can institutions adapt, leverage, and learn from approaches developed during 2020 to attend to challenges experienced by Women of Color in STEMM in the future?

Academic Productivity and Institutional Responses

• How did the institutional responses (e.g., policies, practices) that were outlined in the Major Findings impact women faculty across institutional characteristics and disciplines?
• What are the short- and long-term effects of faculty evaluation practices and extension policies implemented during the COVID-19 pandemic on the productivity and career trajectories of members of the academic STEMM workforce by gender?
• What adaptations did women use during the transition to online and hybrid teaching modes? How did these techniques and adaptations vary as a function of career stage and institutional characteristics?
• What are examples of institutional changes implemented in response to the COVID-19 pandemic that have the potential to reduce systemic barriers to participation and advancement that have historically been faced by academic women in STEMM, specifically Women of Color and other marginalized women in STEMM? How might positive institutional responses be leveraged to create a more resilient and responsive higher education ecosystem?
• How can or should funding arrangements be altered (e.g., changes in funding for research and/or mentorship programs) to support new ways of interaction for women in STEMM during times of disruption, such as the COVID-19 pandemic?

Work-Life Boundaries and Gendered Divisions of Labor

• How do different social identities (e.g., racial; socioeconomic status; culturally, ethnically, sexually, or gender diverse; immigration status; parents of young children and other caregivers; women without partners) influence the management of work-nonwork boundaries? How did this change during the COVID-19 pandemic?
• How have COVID-19 pandemic-related disruptions affected progress toward reducing the gender gap in academic STEMM labor-force participation? How does this differ for Women of Color or women with caregiving responsibilities?
• How can institutions account for the unique challenges of women faculty with parenthood and caregiving responsibilities when developing effective and equitable policies, practices, or programs?
• How might insights gained about work-life boundaries during the COVID-19 pandemic inform how institutions develop and implement supportive resources (e.g., reductions in workload, on-site childcare, flexible working options)?

Collaboration, Networking, and Professional Societies

• What were the short- and long-term effects of the COVID-19 pandemic-prompted switch from in-person conferences to virtual conferences on conference culture and climate, especially for women in STEMM?
• How will the increase in virtual conferences specifically affect women’s advancement and career trajectories? How will it affect women’s collaborations?
• How has the shift away from attending conferences and in-person networking changed longer-term mentoring and sponsoring relationships, particularly in terms of gender dynamics?
• How can institutions maximize the benefits of digitization and the increased use of technology observed during the COVID-19 pandemic to continue supporting women, especially marginalized women, by increasing accessibility, collaborations, mentorship, and learning?
• How can organizations that support, host, or facilitate online and virtual conferences and networking events (1) ensure open and fair access to participants who face different funding and time constraints; (2) foster virtual connections among peers, mentors, and sponsors; and (3) maintain an inclusive environment to scientists of all backgrounds?
• What policies, practices, or programs can be developed to help women in STEMM maintain a sense of support, structure, and stability during and after periods of disruption?

Academic Leadership and Decision-Making

• What specific interventions did colleges and universities initiate or prioritize to ensure that women were included in decision-making processes during responses to the COVID-19 pandemic?
• How effective were colleges and universities that prioritized equity-minded leadership, shared leadership, and crisis leadership styles at mitigating emerging and potential negative effects of the COVID-19 pandemic on women in their communities?
• What specific aspects of different leadership models translated to more effective strategies to advance women in STEMM, particularly during the COVID-19 pandemic?
• How can examples of intentional inclusion of women in decision-making processes during the COVID-19 pandemic be leveraged to develop the engagement of women as leaders at all levels of academic institutions?
• What are potential “top-down” structural changes in academia that can be implemented to mitigate the adverse effects of the COVID-19 pandemic or other disruptions?
• How can academic leadership, at all levels, more effectively support the mental health needs of women in STEMM?

Mental Health and Well-being

• What is the impact of the COVID-19 pandemic and institutional responses on the mental health and well-being of members of the academic STEMM workforce as a function of gender, race, and career stage?
• How are tools and diagnostic tests to measure aspects of wellbeing, including burnout and insomnia, used in academic settings? How does this change during times of increased stress, such as the COVID-19 pandemic?
• How might insights gained about mental health during the COVID-19 pandemic be used to inform preparedness for future disruptions?
• How can programs that focus on changes in biomarkers of stress and mood dysregulation, such as levels of sleep, activity, and texting patterns, be developed and implemented to better engage women in addressing their mental health?
• What are effective interventions to address the health of women academics in STEMM that specifically account for the effects of stress on women? What are effective interventions to mitigate the excessive levels of stress for Women of Color?

This page intentionally left blank.

The spring of 2020 marked a change in how almost everyone conducted their personal and professional lives, both within science, technology, engineering, mathematics, and medicine (STEMM) and beyond. The COVID-19 pandemic disrupted global scientific conferences and individual laboratories and required people to find space in their homes from which to work. It blurred the boundaries between work and non-work, infusing ambiguity into everyday activities. While adaptations that allowed people to connect became more common, the evidence available at the end of 2020 suggests that the disruptions caused by the COVID-19 pandemic endangered the engagement, experience, and retention of women in academic STEMM, and may roll back some of the achievement gains made by women in the academy to date.

The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine identifies, names, and documents how the COVID-19 pandemic disrupted the careers of women in academic STEMM during the initial 9-month period since March 2020 and considers how these disruptions - both positive and negative - might shape future progress for women. This publication builds on the 2020 report Promising Practices for Addressing the Underrepresentation of Women in Science, Engineering, and Medicine to develop a comprehensive understanding of the nuanced ways these disruptions have manifested. The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine will inform the academic community as it emerges from the pandemic to mitigate any long-term negative consequences for the continued advancement of women in the academic STEMM workforce and build on the adaptations and opportunities that have emerged.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The impact of the COVID-19 pandemic on scientific research in the life sciences

Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

Affiliation AXES, IMT School for Advanced Studies Lucca, Lucca, Italy

Roles Conceptualization, Data curation, Formal analysis, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Chair of Systems Design D-MTEC, ETH Zürich, Zurich, Switzerland

• Massimo Riccaboni, 
• Luca Verginer

• Published: February 9, 2022
• https://doi.org/10.1371/journal.pone.0263001
• Reader Comments

The COVID-19 outbreak has posed an unprecedented challenge to humanity and science. On the one side, public and private incentives have been put in place to promptly allocate resources toward research areas strictly related to the COVID-19 emergency. However, research in many fields not directly related to the pandemic has been displaced. In this paper, we assess the impact of COVID-19 on world scientific production in the life sciences and find indications that the usage of medical subject headings (MeSH) has changed following the outbreak. We estimate through a difference-in-differences approach the impact of the start of the COVID-19 pandemic on scientific production using the PubMed database (3.6 Million research papers). We find that COVID-19-related MeSH terms have experienced a 6.5 fold increase in output on average, while publications on unrelated MeSH terms dropped by 10 to 12%. The publication weighted impact has an even more pronounced negative effect (-16% to -19%). Moreover, COVID-19 has displaced clinical trial publications (-24%) and diverted grants from research areas not closely related to COVID-19. Note that since COVID-19 publications may have been fast-tracked, the sudden surge in COVID-19 publications might be driven by editorial policy.

Citation: Riccaboni M, Verginer L (2022) The impact of the COVID-19 pandemic on scientific research in the life sciences. PLoS ONE 17(2): e0263001. https://doi.org/10.1371/journal.pone.0263001

Editor: Florian Naudet, University of Rennes 1, FRANCE

Received: April 28, 2021; Accepted: January 10, 2022; Published: February 9, 2022

Copyright: © 2022 Riccaboni, Verginer. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The processed data, instructions on how to process the raw PubMed dataset as well as all code are available via Zenodo at https://doi.org/10.5281/zenodo.5121216 .

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The COVID-19 pandemic has mobilized the world scientific community in 2020, especially in the life sciences [ 1 , 2 ]. In the first three months after the pandemic, the number of scientific papers about COVID-19 was fivefold the number of articles on H1N1 swine influenza [ 3 ]. Similarly, the number of clinical trials related to COVID-19 prophylaxis and treatments skyrocketed [ 4 ]. Thanks to the rapid mobilization of the world scientific community, COVID-19 vaccines have been developed in record time. Despite this undeniable success, there is a rising concern about the negative consequences of COVID-19 on clinical trial research, with many projects being postponed [ 5 – 7 ]. According to Evaluate Pharma, clinical trials were one of the pandemic’s first casualties, with a record number of 160 studies suspended for reasons related to COVID-19 in April 2020 [ 8 , 9 ] reporting a total of 1,200 trials suspended as of July 2020. As a consequence, clinical researchers have been impaired by reduced access to healthcare research infrastructures. Particularly, the COVID-19 outbreak took a tall on women and early-career scientists [ 10 – 13 ]. On a different ground, Shan and colleagues found that non-COVID-19-related articles decreased as COVID-19-related articles increased in top clinical research journals [ 14 ]. Fraser and coworker found that COVID-19 preprints received more attention and citations than non-COVID-19 preprints [ 1 ]. More recently, Hook and Porter have found some early evidence of ‘covidisation’ of academic research, with research grants and output diverted to COVID-19 research in 2020 [ 15 ]. How much should scientists switch their efforts toward SARS-CoV-2 prevention, treatment, or mitigation? There is a growing consensus that the current level of ‘covidisation’ of research can be wasteful [ 4 , 5 , 16 ].

Against this background, in this paper, we investigate if the COVID-19 pandemic has induced a shift in biomedical publications toward COVID-19-related scientific production. The objective of the study is to show that scientific articles listing covid-related Medical Subject Headings (MeSH) when compared against covid-unrelated MeSH have been partially displaced. Specifically, we look at several indicators of scientific production in the life sciences before and after the start of the COVID-19 pandemic: (1) number of papers published, (2) impact factor weighted number of papers, (3) opens access, (4) number of publications related to clinical trials, (5) number of papers listing grants, (6) number of papers listing grants existing before the pandemic. Through a natural experiment approach, we analyze the impact of the pandemic on scientific production in the life sciences. We consider COVID-19 an unexpected and unprecedented exogenous source of variation with heterogeneous effects across biomedical research fields (i.e., MeSH terms).

Based on the difference in difference results, we document the displacement effect that the pandemic has had on several aspects of scientific publishing. The overall picture that emerges from this analysis is that there has been a profound realignment of priorities and research efforts. This shift has displaced biomedical research in fields not related to COVID-19.

The rest of the paper is structured as follows. First, we describe the data and our measure of relatedness to COVID-19. Next, we illustrate the difference-in-differences specification we rely on to identify the impact of the pandemic on scientific output. In the results section, we present the results of the difference-in-differences and network analyses. We document the sudden shift in publications, grants and trials towards COVID-19-related MeSH terms. Finally, we discuss the findings and highlight several policy implications.

Materials and methods

The present analysis is based primarily on PubMed and the Medical Subject Headings (MeSH) terminology. This data is used to estimate the effect of the start of the COVID 19 pandemic via a difference in difference approach. This section is structured as follows. We first introduce the data and then the econometric methodology. This analysis is not based on a pre-registered protocol.

Selection of biomedical publications.

We rely on PubMed, a repository with more than 34 million biomedical citations, for the analysis. Specifically, we analyze the daily updated files up to 31/06/2021, extracting all publications of type ‘Journal Article’. For the principal analysis, we consider 3,638,584 papers published from January 2019 to December 2020. We also analyze 11,122,017 papers published from 2010 onwards to identify the earliest usage of a grant and infer if it was new in 2020. We use the SCImago journal ranking statistics to compute the impact factor weighted number (IFWN) of papers in a given field of research. To assign the publication date, we use the ‘electronically published’ dates and, if missing, the ‘print published’ dates.

Medical subject headings.

We rely on the Medical Subject Headings (MeSH) terminology to approximate narrowly defined biomedical research fields. This terminology is a curated medical vocabulary, which is manually added to papers in the PubMed corpus. The fact that MeSH terms are manually annotated makes this terminology ideal for classification purposes. However, there is a delay between publication and annotation, on the order of several months. To address this delay and have the most recent classification, we search for all 28 425 MeSH terms using PubMed’s ESearch utility and classify paper by the results. The specific API endpoint is https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi , the relevant scripts are available with the code. For example, we assign the term ‘Ageusia’ (MeSH ID D000370) to all papers listed in the results of the ESearch API. We apply this method to the whole period (January 2019—December 2020) and obtain a mapping from papers to the MeSH terms. For every MeSH term, we keep track of the year they have been established. For instance, COVID-19 terms were established in 2020 (see Table 1 ): in January 2020, the WHO recommended 2019-nCoV and 2019-nCoV acute respiratory disease as provisional names for the virus and disease. The WHO issued the official terms COVID-19 and SARS-CoV-2 at the beginning of February 2020. By manually annotating publications, all publications referring to COVID-19 and SARS-CoV-2 since January 2020 have been labelled with the related MeSH terms. Other MeSH terms related to COVID-19, such as coronavirus, for instance, have been established years before the pandemic (see Table 2 ). We proxy MeSH term usage via search terms using the PubMed EUtilities API; this means that we are not using the hand-labelled MeSH terms but rather the PubMed search results. This means that the accuracy of the MeSH term we assign to a given paper is not perfect. In practice, this means that we have assigned more MeSH terms to a given term than a human annotator would have.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0263001.t001

The list contains only terms with at least 100 publications in 2020.

https://doi.org/10.1371/journal.pone.0263001.t002

Clinical trials and publication types.

We classify publications using PubMed’s ‘PublicationType’ field in the XML baseline files (There are 187 publication types, see https://www.nlm.nih.gov/mesh/pubtypes.html ). We consider a publication to be related to a clinical trial if it lists any of the following descriptors:

• D016430: Clinical Trial
• D017426: Clinical Trial, Phase I
• D017427: Clinical Trial, Phase II
• D017428: Clinical Trial, Phase III
• D017429: Clinical Trial, Phase IV
• D018848: Controlled Clinical Trial
• D065007: Pragmatic Clinical Trial
• D000076362: Adaptive Clinical Trial
• D000077522: Clinical Trial, Veterinary

In our analysis of the impact of COVID-19 on publications related to clinical trials, we only consider MeSH terms that are associated at least once with a clinical trial publication over the two years. We apply this restriction to filter out MeSH terms that are very unlikely to be relevant for clinical trial types of research.

Open access.

We proxy the availability of a journal article to the public, i.e., open access, if it is available from PubMed Central. PubMed Central archives full-text journal articles and provides free access to the public. Note that the copyright license may vary across participating publishers. However, the text of the paper is for all effects and purposes freely available without requiring subscriptions or special affiliation.

We infer if a publication has been funded by checking if it lists any grants. We classify grants as either ‘old’, i.e. existed before 2019, or ‘new’, i.e. first observed afterwards. To do so, we collect all grant IDs for 11,122,017 papers from 2010 on-wards and record their first appearance. This procedure is an indirect inference of the year the grant has been granted. The basic assumption is that if a grant number has not been listed in any publication since 2010, it is very likely a new grant. Specifically, an old grant is a grant listed since 2019 observed at least once from 2010 to 2018.

Note that this procedure is only approximate and has a few shortcomings. Mistyped grant numbers (e.g. ‘1234-M JPN’ and ‘1234-M-JPN’) could appear as new grants, even though they existed before, or new grants might be classified as old grants if they have a common ID (e.g. ‘Grant 1’). Unfortunately, there is no central repository of grant numbers and the associated metadata; however, there are plans to assign DOI numbers to grants to alleviate this problem (See https://gitlab.com/crossref/open_funder_registry for the project).

Impact factor weighted publication numbers (IFWN).

In our analysis, we consider two measures of scientific output. First, we simply count the number of publications by MeSH term. However, since journals vary considerably in terms of impact factor, we also weigh the number of publications by the impact factor of the venue (e.g., journal) where it was published. Specifically, we use the SCImago journal ranking statistics to weigh a paper by the impact factor of the journal it appears in. We use the ‘citation per document in the past two years’ for 45,230 ISSNs. Note that a journal may and often has more than one ISSN, i.e., one for the printed edition and one for the online edition. SCImago applies the same score for a venue across linked ISSNs.

For the impact factor weighted number (IFWN) of publication per MeSH terms, this means that all publications are replaced by the impact score of the journal they appear in and summed up.

COVID-19-relatedness.

To measure how closely related to COVID-19 is a MeSH term, we introduce an index of relatedness to COVID-19. First, we identify the focal COVID-19 terms, which appeared in the literature in 2020 (see Table 1 ). Next, for all other pre-existing MeSH terms, we measure how closely related to COVID-19 they end up being.

Our aim is to show that MeSH terms that existed before and are related have experienced a sudden increase in the number of (impact factor weighted) papers.

Intuitively we can read this measure as: what is the probability in 2020 that a COVID-19 MeSH term is present given that we chose a paper with MeSH term i ? For example, given that in 2020 we choose a paper dealing with “Ageusia” (i.e., Complete or severe loss of the subjective sense of taste), there is a 96% probability that this paper also lists COVID-19, see Table 1 .

Note that a paper listing a related MeSH term does not imply that that paper is doing COVID-19 research, but it implies that one of the MeSH terms listed is often used in COVID-19 research.

In sum, in our analysis, we use the following variables:

• Papers: Number of papers by MeSH term;
• Impact: Impact factor weighted number of papers by MeSH term;
• PMC: Papers listed in PubMed central by MeSH term, as a measure of Open Access publications;
• Trials: number of publications of type “Clinical Trial” by MeSH term;
• Grants: number of papers with at least one grant by MeSH term;
• Old Grants: number of papers listing a grant that has been observed between 2010 and 2018, by MeSH term;

Difference-in-differences

The difference-in-differences (DiD) method is an econometric technique to imitate an experimental research design from observation data, sometimes referred to as a quasi-experimental setup. In a randomized controlled trial, subjects are randomly assigned either to the treated or the control group. Analogously, in this natural experiment, we assume that medical subject headings (MeSH) have been randomly assigned to be either treated (related) or not treated (unrelated) by the pandemic crisis.

Before the COVID, for a future health crisis, the set of potentially impacted medical knowledge was not predictable since it depended on the specifics of the emergency. For instance, ageusia (loss of taste), a medical concept existing since 1991, became known to be a specific symptom of COVID-19 only after the pandemic.

Specifically, we exploit the COVID-19 as an unpredictable and exogenous shock that has deeply affected the publication priorities for biomedical scientific production, as compared to the situation before the pandemic. In this setting, COVID-19 is the treatment, and the identification of this new human coronavirus is the event. We claim that treated MeSH terms, i.e., MeSH terms related to COVID-19, have experienced a sudden increase in terms of scientific production and attention. In contrast, research on untreated MeSH terms, i.e., MeSH terms not related to COVID-19, has been displaced by COVID-19. Our analysis compares the scientific output of COVID-19 related and unrelated MeSH terms before and after January 2020.

In our case, some of the terms turn out to be related to COVID-19 in 2020, whereas most of the MeSH terms are not closely related to COVID-19.

Thus β 1 identifies the overall effect on the control group after the event, β 2 the difference across treated and control groups before the event (i.e. the first difference in DiD) and finally the effect on the treated group after the event, net of the first difference, β 3 . This last parameter identifies the treatment effect on the treated group netting out the pre-treatment difference.

For the DiD to have a causal interpretation, it must be noted that pre-event, the trends of the two groups should be parallel, i.e., the common trend assumption (CTA) must be satisfied. We will show that the CTA holds in the results section.

To specify the DiD model, we need to define a period before and after the event and assign a treatment status or level of exposure to each term.

Before and after.

The pre-treatment period is defined as January 2019 to December 2019. The post-treatment period is defined as the months from January 2020 to December 2020. We argue that the state of biomedical research was similar in those two years, apart from the effect of the pandemic.

Treatment status and exposure.

The treatment is determined by the COVID-19 relatedness index σ i introduced earlier. Specifically, this number indicates the likelihood that COVID-19 will be a listed MeSH term, given that we observe the focal MeSH term i . To show that the effect becomes even stronger the closer related the subject is, and for ease of interpretation, we also discretize the relatedness value into three levels of treatment. Namely, we group MeSH terms with a σ between, 0% to 20%, 20% to 80% and 80% to 100%. The choice of alternative grouping strategies does not significantly affect our results. Results for alternative thresholds of relatedness can be computed using the available source code. We complement the dichotomized analysis by using the treatment intensity (relatedness measure σ ) to show that the result persists.

Panel regression.

In this work, we estimate a random effects panel regression where the units of analysis are 28 318 biomedical research fields (i.e. MeSH terms) observed over time before and after the COVID-19 pandemic. The time resolution is at the monthly level, meaning that for each MeSH term, we have 24 observations from January 2019 to December 2020.

The outcome variable Y it identifies the outcome at time t (i.e., month), for MeSH term i . As before, P t identifies the period with P t = 0 if the month is before January 2020 and P t = 1 if it is on or after this date. In (3) , the treatment level is measure by the relatedness to COVID-19 ( σ i ), where again the γ 1 identifies pre-trend (constant) differences and δ 1 the overall effect.

In total, we estimate six coefficients. As before, the δ l coefficient identifies the DiD effect.

Verifying the Common Trend Assumption (CTA).

We show that the CTA holds for this model by comparing the pre-event trends of the control group to the treated groups (COVID-19 related MeSH terms). Namely, we show that the pre-event trends of the control group are the same as the pre-event trends of the treated group.

Co-occurrence analysis

To investigate if the pandemic has caused a reconfiguration of research priorities, we look at the MeSH term co-occurrence network. Precisely, we extract the co-occurrence network of all 28,318 MeSH terms as they appear in the 3.3 million papers. We considered the co-occurrence networks of 2018, 2019 and 2020. Each node represents a MeSH term in these networks, and a link between them indicates that they have been observed at least once together. The weight of the edge between the MeSH terms is given by the number of times those terms have been jointly observed in the same publications.

Medical language is hugely complicated, and this simple representation does not capture the intricacies, subtle nuances and, in fact, meaning of the terms. Therefore, we do not claim that we can identify how the actual usage of MeSH terms has changed from this object, but rather that it has. Nevertheless, the co-occurrence graph captures rudimentary relations between concepts. We argue that absent a shock to the system, their basic usage patterns, change in importance (within the network) would essentially be the same from year to year. However, if we find that the importance of terms changes more than expected in 2020, it stands to reason that there have been some significant changes.

To show that that MeSH usage has been affected, we compute for each term in the years 2018, 2019 and 2020 their PageRank centrality [ 17 ]. The PageRank centrality tells us how likely a random walker traversing a network would be found at a given node if she follows the weights of the empirical edges (i.e., co-usage probability). Specifically, for the case of the MeSH co-occurrence network, this number represents how often an annotator at the National Library of Medicine would assign that MeSH term following the observed general usage patterns. It is a simplistic measure to capture the complexities of biomedical research. Nevertheless, it captures far-reaching interdependence across MeSH terms as the measure uses the whole network to determine the centrality of every MeSH term. A sudden change in the rankings and thus the position of MeSH terms in this network suggests that a given research subject has risen as it is used more often with other important MeSH terms (or vice versa).

We then compare the growth for each MeSH i term in g i (2019), i.e. before the the COVID-19 pandemic, with the growth after the event ( g i (2020)).

Publication growth

Changes in output and COVID-19 relatedness

Before we show the regression results, we provide descriptive evidence that publications from 2019 to 2020 have drastically increased. By showing that this growth correlates strongly with a MeSH term’s COVID-19 relatedness ( σ ), we demonstrate that (1) σ captures an essential aspect of the growth dynamics and (2) highlight the meteoric rise of highly related terms.

We look at the year over year growth in the number of the impact weighted number of publications per MeSH term from 2018 to 2019 and 2019 to 2020 as defined in the methods section.

Fig 1 shows the yearly growth of the impact weighted number of publications per MeSH term. By comparing the growth of the number of publications from the years 2018, 2019 and 2020, we find that the impact factor weighted number of publications has increased by up to a factor of 100 compared to the previous year for Betacoronavirus, one of the most closely related to COVID-19 MeSH term.

Each dot represents, a MeSH term. The y axis (growth) is in symmetric log scale. The x axis shows the COVID-19 relatedness, σ . Note that the position of the dots on the x-axis is the same in the two plots. Below: MeSH term importance gain (PageRank) and their COVID-19 relatedness.

https://doi.org/10.1371/journal.pone.0263001.g001

Fig 1 , first row, reveals how strongly correlated the growth in the IFWN of publication is to the term’s COVID-19 relatedness. For instance, we see that the term ‘Betacoronavirus’ skyrocketed from 2019 to 2020, which is expected given that SARS-CoV-2 is a species of the genus. Conversely, the term ‘Alphacoronavirus’ has not experienced any growth given that it is twin a genus of the Coronaviridae family, but SARS-CoV-2 is not one of its species. Note also the fast growth in the number of publications dealing with ‘Quarantine’. Moreover, MeSH terms that grew significantly from 2018 to 2019 and were not closely related to COVID-19, like ‘Vaping’, slowed down in 2020. From the graph, the picture emerges that publication growth is correlated with COVID-19 relatedness σ and that the growth for less related terms slowed down.

To show that the usage pattern of MeSH terms has changed following the pandemic, we compute the PageRank centrality using graph-tool [ 18 ] as discussed in the Methods section.

Fig 1 , second row, shows the change in the PageRank centrality of the MeSH terms after the pandemic (2019 to 2020, right plot) and before (2018 to 2019, left plot). If there were no change in the general usage pattern, we would expect the variance in PageRank changes to be narrow across the two periods, see (left plot). However, PageRank scores changed significantly more from 2019 to 2020 than from 2018 to 2019, suggesting that there has been a reconfiguration of the network.

To further support this argument, we carry out a DiD regression analysis.

Common trends assumption

As discussed in the Methods section, we need to show that the CTA assumption holds for the DiD to be defined appropriately. We do this by estimating for each month the number of publications and comparing it across treatment groups. This exercise also serves the purpose of a placebo test. By assuming that each month could have potentially been the event’s timing (i.e., the outbreak), we show that January 2020 is the most likely timing of the event. The regression table, as noted earlier, contains over 70 estimated coefficients, hence for ease of reading, we will only show the predicted outcome per month by group (see Fig 2 ). The full regression table with all coefficients is available in the S1 Table .

The y axis is in log scale. The dashed vertical line identifies January 2020. The dashed horizontal line shows the publications in January 2019 for the 0–20% group before the event. This line highlights that the drop happens after the event. The bands around the lines indicate the 95% confidence interval of the predicted values. The results are the output of the Stata margins command.

https://doi.org/10.1371/journal.pone.0263001.g002

Fig 2 shows the predicted number per outcome variable obtained from the panel regression model. These predictions correspond to the predicted value per relatedness group using the regression parameters estimated via the linear panel regression. The bands around the curves are the 95% confidence intervals.

All outcome measures depict a similar trend per month. Before the event (i.e., January 2020), there is a common trend across all groups. In contrast, after the event, we observe a sudden rise for the outcomes of the COVID-19 related treated groups (green and red lines) and a decline in the outcomes for the unrelated group (blue line). Therefore, we can conclude that the CTA assumption holds.

Regression results

Table 3 shows the DiD regression results (see Eq (3) ) for the selected outcome measures: number of publications (Papers), impact factor weighted number of publications (Impact), open access (OA) publications, clinical trial related publications, and publications with existing grants.

https://doi.org/10.1371/journal.pone.0263001.t003

Table 3 shows results for the discrete treatment level version of the DiD model (see Eq (4) ).

Note that the outcome variable is in natural log scale; hence to get the effect of the independent variable, we need to exponentiate the coefficient. For values close to 0, the effect is well approximated by the percentage change of that magnitude.

In both specifications we see that the least related group, drops in the number of publications between 10% and 13%, respectively (first row of Tables 3 and 4 , exp(−0.102) ≈ 0.87). In line with our expectations, the increase in the number of papers published by MeSH term is positively affected by the relatedness to COVID-19. In the discrete model (row 2), we note that the number of documents with MeSH terms with a COVID-19 relatedness between 20 and 80% grows by 18% and highly related terms by a factor of approximately 6.6 (exp(1.88)). The same general pattern can be observed for the impact weighted publication number, i.e., Model (2). Note, however, that the drop in the impact factor weighted output is more significant, reaching -19% for COVID-19 unrelated publications, and related publications growing by a factor of 8.7. This difference suggests that there might be a bias to publish papers on COVID-19 related subjects in high impact factor journals.

https://doi.org/10.1371/journal.pone.0263001.t004

By looking at the number of open access publications (PMC), we note that the least related group has not been affected negatively by the pandemic. However, the number of COVID-19 related publications has drastically increased for the most COVID-19 related group by a factor of 6.2. Note that the substantial increase in the number of papers available through open access is in large part due to journal and editorial policies to make preferentially COVID research immediately available to the public.

Regarding the number of clinical trial publications, we note that the least related group has been affected negatively, with the number of publications on clinical trials dropping by a staggering 24%. At the same time, publications on clinical trials for COVID-19-related MeSH have increased by a factor of 2.1. Note, however, that the effect on clinical trials is not significant in the continuous regression. The discrepancy across Tables 3 and 4 highlights that, especially for trials, the effect is not linear, where only the publications on clinical trials closely related to COVID-19 experiencing a boost.

It has been reported [ 19 ] that while the number of clinical trials registered to treat or prevent COVID-19 has surged with 179 new registrations in the second week of April 2020 alone. Only a few of these have led to publishable results in the 12 months since [ 20 ]. On the other hand, we find that clinical trial publications, considering related MeSH (but not COVID-19 directly), have had significant growth from the beginning of the pandemic. These results are not contradictory. Indeed counting the number of clinical trial publications listing the exact COVID-19 MeSH term (D000086382), we find 212 publications. While this might seem like a small number, consider that in 2020 only 8,485 publications were classified as clinical trials; thus, targeted trials still made up 2.5% of all clinical trials in 2020 . So while one might doubt the effectiveness of these research efforts, it is still the case that by sheer number, they represent a significant proportion of all publications on clinical trials in 2020. Moreover, COVID-19 specific Clinical trial publications in 2020, being a delayed signal of the actual trials, are a lower bound estimate on the true number of such clinical trials being conducted. This is because COVID-19 studies could only have commenced in 2020, whereas other studies had a head start. Thus our reported estimates are conservative, meaning that the true effect on actual clinical trials is likely larger, not smaller.

Research funding, as proxied by the number of publications with grants, follows a similar pattern, but notably, COVID-19-related MeSH terms list the same proportion of grants established before 2019 as other unrelated MeSH terms, suggesting that grants which were not designated for COVID-19 research have been used to support COVID-19 related research. Overall, the number of publications listing a grant has dropped. Note that this should be because the number of publications overall in the unrelated group has dropped. However, we note that the drop in publications is 10% while the decline in publications with at least one grant is 15%. This difference suggests that publications listing grants, which should have more funding, are disproportionately COVID-19 related papers. To further investigate this aspect, we look at whether the grant was old (pre-2019) or appeared for the first time in or after 2019. It stands to reason that an old grant (pre-2019) would not have been granted for a project dealing with the pandemic. Hence we would expect that COVID-19 related MeSH terms to have a lower proportion of old grants than the unrelated group. In models (6) in Table 4 we show that the number of old grants for the unrelated group drops by 13%. At the same time, the number of papers listing old grants (i.e., pre-2019) among the most related group increased by a factor of 3.1. Overall, these results suggest that COVID-19 related research has been funded largely by pre-existing grants, even though a specific mandate tied to the grants for this use is unlikely.

The scientific community has swiftly reallocated research efforts to cope with the COVID-19 pandemic, mobilizing knowledge across disciplines to find innovative solutions in record time. We document this both in terms of changing trends in the biomedical scientific output and the usage of MeSH terms by the scientific community. The flip side of this sudden and energetic prioritization of effort to fight COVID-19 has been a sudden contraction of scientific production in other relevant research areas. All in all, we find strong support to the hypotheses that the COVID-19 crisis has induced a sudden increase of research output in COVID-19 related areas of biomedical research. Conversely, research in areas not related to COVID-19 has experienced a significant drop in overall publishing rates and funding.

Our paper contributes to the literature on the impact of COVID-19 on scientific research: we corroborate previous findings about the surge of COVID-19 related publications [ 1 – 3 ], partially displacing research in COVID-19 unrelated fields of research [ 4 , 14 ], particularly research related to clinical trials [ 5 – 7 ]. The drop in trial research might have severe consequences for patients affected by life-threatening diseases since it will delay access to new and better treatments. We also confirm the impact of COVID-19 on open access publication output [ 1 ]; also, this is milder than traditional outlets. On top of this, we provide more robust evidence on the impact weighted effect of COVID-19 and grant financed research, highlighting the strong displacement effect of COVID-19 on the allocation of financial resources [ 15 ]. We document a substantial change in the usage patterns of MeSH terms, suggesting that there has been a reconfiguration in the way research terms are being combined. MeSH terms highly related to COVID-19 were peripheral in the MeSH usage networks before the pandemic but have become central since 2020. We conclude that the usage patterns have changed, with COVID-19 related MeSH terms occupying a much more prominent role in 2020 than they did in the previous years.

We also contribute to the literature by estimating the effect of COVID-19 on biomedical research in a natural experiment framework, isolating the specific effects of the COVID-19 pandemic on the biomedical scientific landscape. This is crucial to identify areas of public intervention to sustain areas of biomedical research which have been neglected during the COVID-19 crisis. Moreover, the exploratory analysis on the changes in usage patterns of MeSH terms, points to an increase in the importance of covid-related topics in the broader biomedical research landscape.

Our results provide compelling evidence that research related to COVID-19 has indeed displaced scientific production in other biomedical fields of research not related to COVID-19, with a significant drop in (impact weighted) scientific output related to non-COVID-19 and a marked reduction of financial support for publications not related to COVID-19 [ 4 , 5 , 16 ]. The displacement effect is persistent to the end of 2020. As vaccination progresses, we highlight the urgent need for science policy to re-balance support for research activity that was put on pause because of the COVID-19 pandemic.

We find that COVID-19 dramatically impacted clinical research. Reactivation of clinical trials activities that have been postponed or suspended for reasons related to COVID-19 is a priority that should be considered in the national vaccination plans. Moreover, since grants have been diverted and financial incentives have been targeted to sustain COVID-19 research leading to an excessive entry in COVID-19-related clinical trials and the ‘covidisation’ of research, there is a need to reorient incentives to basic research and otherwise neglected or temporally abandoned areas of biomedical research. Without dedicated support in the recovery plans for neglected research of the COVID-19 era, there is a risk that more medical needs will be unmet in the future, possibly exacerbating the shortage of scientific research for orphan and neglected diseases, which do not belong to COVID-19-related research areas.

Limitations

Our empirical approach has some limits. First, we proxy MeSH term usage via search terms using the PubMed EUtilities API. This means that the accuracy of the MeSH term we assign to a given paper is not fully validated. More time is needed for the completion of manually annotated MeSH terms. Second, the timing of publication is not the moment the research has been carried out. There is a lead time between inception, analysis, write-up, review, revision, and final publication. This delay varies across disciplines. Nevertheless, given that the surge in publications happens around the alleged event date, January 2020, we are confident that the publication date is a reasonable yet imperfect estimate of the timing of the research. Third, several journals have publicly declared to fast-track COVID-19 research. This discrepancy in the speed of publication of COVID-19 related research and other research could affect our results. Specifically, a surge or displacement could be overestimated due to a lag in the publication of COVID-19 unrelated research. We alleviate this bias by estimating the effect considering a considerable time after the event (January 2020 to December 2020). Forth, on the one hand, clinical Trials may lead to multiple publications. Therefore we might overestimate the impact of COVID-19 on the number of clinical trials. On the other hand, COVID-19 publications on clinical trials lag behind, so the number of papers related COVID-19 trials is likely underestimated. Therefore, we note that the focus of this paper is scientific publications on clinical trials rather than on actual clinical trials. Fifth, regarding grants, unfortunately, there is no unique centralized repository mapping grant numbers to years, so we have to proxy old grants with grants that appeared in publications from 2010 to 2018. Besides, grant numbers are free-form entries, meaning that PubMed has no validation step to disambiguate or verify that the grant number has been entered correctly. This has the effect of classifying a grant as new even though it has appeared under a different name. We mitigate this problem by using a long period to collect grant numbers and catch many spellings of the same grant, thereby reducing the likelihood of miss-identifying a grant as new when it existed before. Still, unless unique identifiers are widely used, there is no way to verify this.

So far, there is no conclusive evidence on whether entry into COVID-19 has been excessive. However, there is a growing consensus that COVID-19 has displaced, at least temporally, scientific research in COVID-19 unrelated biomedical research areas. Even though it is certainly expected that more attention will be devoted to the emergency during a pandemic, the displacement of biomedical research in other fields is concerning. Future research is needed to investigate the long-run structural consequences of the COVID-19 crisis on biomedical research.

Supporting information

S1 table. common trend assumption (cta) regression table..

Full regression table with all controls and interactions.

https://doi.org/10.1371/journal.pone.0263001.s001

• View Article
• Google Scholar
• PubMed/NCBI
• 8. Brown A, Edwin E, Fagg J. Evaluate Pharma 2021 Preview; 2020. https://www.evaluate.com/thought-leadership/vantage/evaluate-vantage-2021-preview .
• 15. Hook D, Porter S. The COVID Brain Drain; 2020. https://www.natureindex.com/news-blog/covid-brain-drain-scientific-research .
• 17. Page L, Brin S, Motwani R, Winograd T. The PageRank citation ranking: Bringing order to the web. Stanford InfoLab; 1999.

COVID-19 symptoms and what to do

Symptoms of covid-19.

COVID-19 symptoms can include:

• a high temperature or shivering (chills) – a high temperature means you feel hot to touch on your chest or back (you do not need to measure your temperature)
• a new, continuous cough – this means coughing a lot for more than an hour, or 3 or more coughing episodes in 24 hours
• a loss or change to your sense of smell or taste
• shortness of breath
• feeling tired or exhausted
• an aching body
• a sore throat
• a blocked or runny nose
• loss of appetite
• feeling sick or being sick

The symptoms are very similar to symptoms of other illnesses, such as colds and flu.

Most people feel better within a few days or weeks of their first COVID-19 symptoms and make a full recovery within 12 weeks. For some people, it can be a more serious illness and their symptoms can last longer.

What to do if you have symptoms of COVID-19

You may be able to look after yourself at home if you have COVID-19 or symptoms of COVID-19 .

Try to stay at home and avoid contact with other people if you or your child have symptoms and either:

• have a high temperature
• do not feel well enough to go to work, school, childcare, or do your normal activities

You can go back to your normal activities when you feel better or do not have a high temperature.

If your child has mild symptoms such as a runny nose, sore throat or mild cough, and they feel well enough, they can go to school or childcare.

If you are eligible for treatments for COVID-19, you should take a rapid lateral flow test as soon as you get symptoms.

Find out more about treatments for COVID-19 and who can have them

What to do if you have tested positive

You are no longer required to do a COVID-19 rapid lateral flow test if you have symptoms.

But if you or your child have tested positive for COVID-19:

• try to stay at home and avoid contact with other people for 3 days after the day the test was taken if you or your child are under 18 years old – children and young people tend to be infectious to other people for less time than adults
• try to stay at home and avoid contact with other people for 5 days after the day you took your test if you are 18 years old or over
• avoid meeting people who are more likely to get seriously ill from viruses, such as people with a weakened immune system, for 10 days after the day you took your test

Find out more about who can get a free COVID-19 test

Urgent advice: Ask for an urgent GP appointment or get help from NHS 111 if:

• you're worried about your or a child's COVID-19 symptoms or are not sure what to do
• the symptoms are getting worse or are not getting better
• you or a child have other signs of illness, such as a rash, loss of appetite, or feeling weak
• you or a child have a high temperature that last 5 days or more or does not come down with paracetamol
• a child under 3 months old and has a temperature of 38C or higher, or you think they have a high temperature
• a child 3 to 6 months old and has a temperature of 39C or higher, or you think they have a high temperature

It's particularly important to get help if you're at increased risk of getting ill from COVID-19, such as if you're pregnant, aged 60 or over, or have a weakened immune system.

You can call 111 or get help from 111 online .

Immediate action required: Call 999 or go to A&E if you or a child:

• seems very unwell, is getting worse or you think there's something seriously wrong – children and babies in particular can get unwell very quickly
• get sudden chest pain
• are so breathless you're unable to say short sentences when resting or your breathing has suddenly got worse – in babies their stomach may suck in under their ribs
• start coughing up blood
• collapse, faint, or have a seizure or fit for the first time
• a rash that does not fade when you roll a glass over it, the same as meningitis

More information

Read full guidance for people with symptoms of a respiratory infection including COVID-19 on GOV.UK

GOV.UK guidance for people whose immune system means they are at higher risk from COVID-19

Page last reviewed: 21 March 2023 Next review due: 21 March 2026

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 September 2021

News media coverage of COVID-19 public health and policy information

• Katharine J. Mach 1 , 2 ,
• Raúl Salas Reyes   ORCID: orcid.org/0000-0002-1683-8516 3 ,
• Brian Pentz   ORCID: orcid.org/0000-0003-2713-6699 3 ,
• Jennifer Taylor   ORCID: orcid.org/0000-0002-8301-3434 4 ,
• Clarissa A. Costa 3 ,
• Sandip G. Cruz 3 ,
• Kerronia E. Thomas 3 ,
• James C. Arnott   ORCID: orcid.org/0000-0003-3989-6724 5 ,
• Rosalind Donald 1 ,
• Kripa Jagannathan   ORCID: orcid.org/0000-0003-4584-8358 6 , 7 ,
• Christine J. Kirchhoff   ORCID: orcid.org/0000-0002-2686-6764 8 ,
• Laura C. Rosella   ORCID: orcid.org/0000-0003-4867-869X 9 &
• Nicole Klenk   ORCID: orcid.org/0000-0001-8224-6992 3  

Humanities and Social Sciences Communications volume  8 , Article number:  220 ( 2021 ) Cite this article

38k Accesses

55 Citations

72 Altmetric

Metrics details

• Cultural and media studies
• Science, technology and society

During a pandemic, news media play a crucial role in communicating public health and policy information. Traditional newspaper coverage is important amidst increasing disinformation, yet uncertainties make covering health risks and efforts to limit transmission difficult. This study assesses print and online newspaper coverage of the coronavirus disease COVID-19 for March 2020, when the global pandemic was declared, through August 2020 in three countries: Canada (with the lowest per-capita case and death rates during the study timeframe), the United Kingdom (with a pronounced early spike), and the United States (with persistently high rates). Tools previously validated for pandemic-related news records allow measurement of multiple indicators of scientific quality (i.e., reporting that reflects the state of scientific knowledge) and of sensationalism (i.e., strategies rendering news as more extraordinary than it really is). COVID-19 reporting had moderate scientific quality and low sensationalism across 1331 sampled articles in twelve newspapers spanning the political spectrums of the three countries. Newspapers oriented towards the populist-right had the lowest scientific quality in reporting, combined with very low sensationalism in some cases. Against a backdrop of world-leading disease rates, U.S. newspapers on the political left had more exposing coverage, e.g., focused on policy failures or misinformation, and more warning coverage, e.g., focused on the risks of the disease, compared to U.S. newspapers on the political right. Despite the generally assumed benefits of low sensationalism, pandemic-related coverage with low scientific quality that also failed to alert readers to public-health risks, misinformation, or policy failures may have exacerbated the public-health effects of the disease. Such complexities will likely remain central for both pandemic news media reporting and public-health strategies reliant upon it.

Similar content being viewed by others

Newspapers’ coverage of the COVID-19 pandemic in Eswatini: from distanciated re/presentations to socio-health panics

Spread of awareness of COVID-19 between December 2019 and March 2020 in France

Anti-intellectualism and the mass public’s response to the COVID-19 pandemic

Introduction.

News media reporting is understood to play a central role during national security and health emergencies (Laing, 2011 ; Klemm et al., 2016 ; Pieri, 2019 ). News coverage communicates risks to readers and shapes public perceptions through the amount, content, and tone of reporting. It simultaneously frames ongoing public debates about policy responses, including conflicting priorities relevant to the timing or stringency of implemented policies (Laing, 2011 ; Pieri, 2019 ). Pandemic policy-making requires rapid, iterative responses under conditions of knowledge deficit, as well as the coordination of multi-level public-health agencies and sectors (e.g., hospitals, schools, and workplaces) (Laing, 2011 ; Rosella et al., 2013 ). In these complex circumstances, news media serve as a primary source of health information and uncertainties and connect health professionals, policymakers, and the public in critical ways (Laing, 2011 ; Hoffman and Justicz, 2016 ). The quality and balance of scientific coverage, such as through reporting that reflects the state of scientific knowledge and is not overstated, affect trust in science and accountability for decision-making (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ).

Inadequate scientific quality in news coverage of past pandemics has posed risks and limited capacities to disseminate public-health guidance and coordinate responses (Hoffman and Justicz, 2016 ). Reporting on the state of scientific knowledge during a novel, evolving pandemic is challenging. Low-quality scientific reporting of pandemics may overstate or understate disease risks or the efficacy of protective measures for different individuals or fail to communicate the nature of the evidence. Such reporting may constrain the feasibility or effectiveness of options for policymakers directing government action, miss opportunities to inform individuals making health decisions, and increase the exposure of health professionals to disease. It can both exacerbate disease outcomes and generate unnecessary fear, in combination with other factors shaping perceptions among the public (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ). For example, news media reporting may have overly emphasized the threat of the 2009 A/H1N1 influenza (H1N1) pandemic with insufficient indication of available protective measures, and in pairing trustworthy information from credible scientists with uninformed opinions, it may have promoted a “false balance” (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ). Further, news coverage rapidly waned after the initial pandemic declaration even though public-health risks persisted (Klemm et al., 2016 ; Reintjes et al., 2016 ). Similar issues with media reporting occurred during the 2003 severe acute respiratory syndrome (SARS) outbreak and the 2014 Ebola outbreak (Hoffman and Justicz, 2016 ; Pieri, 2019 ).

During the COVID-19 pandemic, media representations of complex, rapidly evolving epidemiological science shape public understandings of the risks, measures to limit disease spread, and associated political and policy discourses. Traditional newspaper media coverage may have particular importance given simultaneous misinformation and disinformation, social fragmentation, political polarization, and failures of policy coordination, and national newspapers influence how other outlets cover the same subject across media platforms (Ball and Maxmen, 2020 ; Holtz et al., 2020 ; Thorp, 2020 ; Grossman et al., 2020 ). The COVID-19 pandemic creates an opportunity to assess the strengths and limitations of the media’s pandemic coverage and provide insights for future news media coverage. Such assessment also informs the communication strategies of public-health institutions and policymakers towards clear public-health guidance and coordinated responses across health systems (Laing, 2011 ; Hoffman and Justicz, 2016 ; Pieri, 2019 ).

Canada, the United Kingdom, and the United States, our countries of focus, differ in how they govern public health, including pandemic responses. In its constitutionally determined role, the Canadian federal government sets healthcare standards and administers funding to support the healthcare system spanning provinces and territories (Government of Canada, 2016 ). Pandemic health-related policies are set and implemented predominantly by provinces with federal guidance from Health Canada and the Public Health Agency of Canada (Adeel et al., 2020 ). The U.K. central government funds healthcare throughout the United Kingdom yet only sets policies for England. Northern Ireland, Scotland, and Wales each govern their own National Health Service systems. By contrast, the healthcare system in the United States is a complex mixture of public and private health insurance programs. The U.S. federal government generally adopts a leading role during national crises, although during the COVID-19 pandemic states and municipalities have led adoption and implementation of most policy measures to contain the spread of COVID-19 (Adeel et al., 2020 ). Prior to the COVID-19 pandemic, the 2019 Global Health Security Index ranked the United States first, United Kingdom second, and Canada fifth among 195 countries for preparedness to manage a serious disease outbreak (Cameron et al., 2019 ).

In this paper, we systematically quantify the amount, scientific quality, and sensationalism of newspaper media coverage of COVID-19 in Canada, the United Kingdom, and the United States. Newspapers studied span the political spectrum of each case-study country (Table 1 ) (Gentzkow and Shapiro, 2010 ; Puglisi and Snyder, 2015 ; Anderson and Coletto, 2017 ; Mitchell et al., 2018 ; Hönnige et al., 2020 ; Jurkowitz et al., 2020 ; Austen, 2020 ). Our analysis begins two weeks prior to COVID-19’s official recognition as a pandemic and follows its development over the subsequent five months (i.e., from 1 March 2020 to 15 August 2020). Given the volume of COVID-19 news media articles published over the timeframe of this study, we created a manageable corpus for analysis by randomly sampling one day of media coverage per week for six consecutive 4-week periods; we then randomly selected five eligible articles from each news outlet on each sampled day for the evaluation of scientific quality and sensationalism. In our evaluation, scientific quality refers to the alignment between reporting and the state of scientific evidence and its uncertainties, and sensationalism is a discursive strategy rendering news as more extraordinary, interesting, or relevant than it really is (Oxman et al., 1993 ; Molek-Kozakowska, 2013 ; Hoffman and Justicz, 2016 ). We apply previously validated survey tools developed to measure scientific quality and sensationalism of pandemic-related health news records in combination with broader methods from policy analyses of pandemic responses (SI Coding Tool) (Oxman et al., 1993 ; Rosella et al., 2013 ; Molek-Kozakowska, 2013 ; Reintjes et al., 2016 ; Hoffman and Justicz, 2016 ). We analyze (1) the COVID-19 public-health outcomes and policies in each country and (2) the amount, scientific quality, sensationalism, and topics of COVID-19 news media coverage across the political spectrum of each country.

Public health contextualization of news media analyses

To contextualize our news media analyses, we analyzed and visualized existing data sets on the number of COVID-19 cases, deaths, and tests in each country (e.g., Roser et al., 2020 ; CBC News, 2020 ; Public Health England and NHSX, 2020 ; CDC, 2020 ). We also recorded the key public-health declarations, policies, and guidance during the study time period (e.g., drawing from WHO, 2020a , 2020b ; see also SI Table S1 ). We tracked these decisions at international scales through to subnational scales in each country studied. Media analyses outlined below thereby were considered with respect to the reported number of cases and confirmed deaths and policy actions taken (Reintjes et al., 2016 ).

News media search strategy and inclusion criteria

Print and online news media records were retrieved from the Factiva database for news outlets across the political spectrum of Canada, the United Kingdom, and the United States (see Table 1 ) (Gentzkow and Shapiro, 2010 ; Puglisi and Snyder, 2015 ; Anderson and Coletto, 2017 ; Mitchell et al., 2018 ; Hönnige et al., 2020 ; Jurkowitz et al., 2020 ; Austen, 2020 ). Selected news media outlets have primary news products in print and online media, rather than television broadcasting or social media, and full article entries available in Factiva. Search terms included “coronavirus,” “COVID-19,” “epidemic,” “outbreak,” “pandemic,” or “SARS-CoV-2.” Individual English-language news articles were retrieved for sampled dates between 1 March 2020 and 15 August 2020. This period captures news media coverage prior to the declaration of the COVID-19 pandemic and over the subsequent five months.

Individual news records were screened to identify original news reporting (i.e., news reporting and news analysis articles) relevant to our study objectives. First, eligible articles must have a direct focus on the public-health implications of COVID-19 or on attempts to control its spread—in some or all of an article’s text. By excluding articles without this focus, we ensured all articles included in the study could contain scientific information on the public health effects or spread of COVID-19 and associated policies. Second, eligible articles must be focused on the newspaper’s country of publication (e.g., an article reporting on COVID-19 transmission or mitigation efforts in only New Zealand or China, without discussion of implications for the newspaper’s country of publication, would be excluded). We included this eligibility criterion to analyze science–policy interfaces and science–society interactions most proximate to the news outlets, although we acknowledge that articles about other countries may influence perceptions of readers even without direct discussion of implications for them. Third, eligible articles must be original news reporting or analysis, meaning we excluded opinion pieces, editorials, interview transcripts, microblogs, front-page snippets, news roundups, obituaries, advertisements, corrections memos, and letters to the editor; these excluded article types would have required distinct question framings beyond the scope of our codebook. This third criterion, therefore, ensured that coded responses could be compared coherently across articles for the different measures of scientific quality and sensationalism.

Sampling of news media articles

As the evaluation of scientific quality and sensationalism through manual coding is time intensive, and a very large number of COVID-19 news media articles were published during the timeframe of our study, we used a random sample of news media articles for analysis, prioritizing sampling during each week over the course of the study timeframe. The sample design enabled a manageable analysis of newspaper media coverage and potential changes over the timeframe of the study. First, the sample of news media articles was constructed by sampling one day of media coverage per week in consecutive four-week periods. These four days of the week were randomly sampled without replacement (Monday through Saturday only, not including Sunday in the sampling), given cyclic variation in news media coverage (Lacy et al., 2001 ; Riffe et al., 2016 ). The study timeframe was divided into six four-week periods of equal duration from 1 March to 15 August 2020.

Second, for each randomly sampled day, all available news records were retrieved from Factiva for the 12 news outlets (Table 1 ). Randomly selected articles were screened for eligibility, with the goal of identifying 5 eligible articles for each news outlet on each sampled day. In some cases, fewer than 5 eligible articles were published by a given outlet on a sampled day. In these cases, the full set of eligible articles was included in the study.

Analysis of scientific quality and sensationalism of news articles

The coding tool for measuring scientific quality and sensationalism of news article records was adapted from the final tool of Hoffman and Justicz, designed for evaluating pandemic-related health news records (Hoffman and Justicz, 2016 ). Scientific quality, as defined in that study, is “a measure of an article’s reliability and credibility on a given topic” (Hoffman and Justicz, 2016 ). Importantly, scientific quality is linked to the state of scientific understanding and its uncertainties at specific moments in time rather than being an absolute or objective characteristic. The codebook we applied for measuring scientific quality is therefore designed to be flexible and responsive to the inevitable shifts in scientific understanding that occur through time, most especially during a novel disease outbreak and evolving pandemic. Sensationalism, as defined in that study, is “a way of presenting articles to make them seem more interesting or extraordinary than they actually are” (Hoffman and Justicz, 2016 ).

Our coding tool (SI Coding Tool) included six questions for scientific quality (each evaluated on a scale from 1 to 5—5 corresponding to highest quality) and six questions for sensationalism (each evaluated on a scale from 1 to 5—5 corresponding to highest sensationalism). The question categories (SI Coding Tool) for assessing scientific quality were as follows: applicability, opinion versus facts, validity, precision, context, and global assessment (i.e., an overall assessment of the article’s scientific quality based on the five preceding specific measures). For sensationalism, the question categories (SI Coding Tool) included exposing, speculating, generalizing, warning, extolling, and global assessment (i.e., an overall assessment of the degree of sensationalism in the article based on the five preceding specific measures) (Oxman et al., 1993 ; Molek-Kozakowska, 2013 ; Hoffman and Justicz, 2016 ).

In addition, metadata collected for each article included the coder’s identity, the article title, the article’s sample date, the news outlet (including if the article was originally written by another outlet such as the Associated Press), the societal sector (up to 2 selected per article), and public-health measures discussed (SI Coding Tool).

Coder assignments, training, reliability assessment, and analysis

For each sampled day, two independent coders assessed all relevant news media records based on the scientific quality and sensationalism questions and article-attribute metadata. Coders recorded scores for each article through a Google-form version of the codebook (SI Coding Tool).

To ensure consistent application of the coding tool, substantial training and calibration occurred over a six-week period. First, the three coders in coordination with the project leadership team read national and international public-health agency descriptions of the coronavirus disease and associated public-health policies and measures. Second, the coders completed multiple rounds of individual coding of example news articles, followed by group discussions of application of the codebook. The group discussions considered difficult judgments and common versus unusual examples. The goal was to ensure consistent application of the coding tool across question categories and the range of article examples that arose. During the training and calibration phases of coding, we updated the codebook to include examples specific to news records on COVID-19 (SI Coding Tool), and we tracked illustrative examples (news articles and specific quotes) across the scale (1–3–5) for the scientific quality and sensationalism question categories. This process led to development of example answers particularly representative of low versus high scientific quality and low versus high sensationalism under each category of response. Additionally, we developed “decision rules” for the more unusual or challenging categories of examples to ensure consistency across coders, especially where disagreements arose in individually assigned responses.

Interrater reliability was assessed during the training and calibration stage and throughout the duration of the study. Where coders assigned scores for a given question that were 3 or 4 units apart on the 1–5 scale, a reconciliation discussion occurred; the small fraction of question responses in this category following the training stage enabled the coders and project team to continue developing and ensuring shared understanding of coding approaches for unusual or challenging applications. Weighted Cohen’s Kappa, with quadratic weighting, was applied given the high-inference codebook and ordinal data collected via a Likert scale, as previously done for related measures (Cohen, 1960 ; Fleiss and Cohen, 1973 ; Oxman et al., 1993 ; Antoine et al., 2014 ; Hoffman and Justicz, 2016 ; Tran et al., 2020 ). Coded data were analyzed with Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison pairwise tests (kruskal.test and kruskalmc in pgirmess package in R) (Giraudoux et al., 2018 ; R Core Team, 2020 ).

Public health and policy contexts

From March through August 2020, Canada, the United Kingdom, and the United States differed substantially in their public-health responses to COVID-19 and in health outcomes from the novel coronavirus disease (Fig. 1 ). Beginning in early March, all three countries implemented a combination of policy measures to contain the spread of COVID-19, including emergency laws, stay-at-home orders, mask mandates, school and business closures, border and travel restrictions, social distancing measures, and quarantines upon entry (SI Table S1 ). These restrictions were followed by gradual phases of reopening measures allowing restricted social and economic activities to occur. Across the three countries, the role of national versus subnational governments differed with respect to authority and actions on public-health guidance and care, resulting in differing timing and levels of coordination for both restrictions and reopening measures (SI Fig. S1 ). From March to August 2020, the United Kingdom experienced the highest death rate from COVID-19 (maximum 7-day average of 13.9 deaths per million people; Fig. 1 ), whereas the United States had the highest case rate of the three countries (maximum 7-day average of 203.5 cases per million), as well as the greatest cumulative number of cases and deaths globally (SI Figs. S2 - S3 ). Of the three countries, Canada had the most effective public-health outcomes as measured by per capita COVID-19 case or death rates (Fig. 1 ).

COVID-19 cases, deaths, and national-level policies are indicated for ( A ) Canada, ( B ) the United Kingdom, and ( C ) the United States. 7-day rolling averages of cases (left vertical axis, solid black line) and deaths (right vertical axis, dotted black line) per one million people are shown for the timeframe of this media study, 1 March through 15 August 2020 (Roser et al., 2020 ). The timeline for each country specifies national-level public-health policies and guidance, especially emergency declarations, school and non-essential business closures, travel and border restrictions, quarantines and social distancing, mask usage, and reopening phases. Implementation of enforceable policies (solid) and non-enforced guidance (dotted) is specified with vertical red lines, and corresponding reopening and relaxation of policies and guidance are specified with vertical blue lines. Detailed descriptions of national-level policies within each panel are provided in SI Table S1 .

The amount of pandemic media coverage

The studied news outlets differed in the amount of news media coverage related to COVID-19 from 1 March through 15 August 2020 (Fig. 2 ). The amount of coverage increased notably in March as case rates climbed in each country, subsequently decreasing gradually in May and June while case rates also declined. Across the 24 randomly sampled days, the 12 studied news outlets published 18,430 articles related to COVID-19. Of these, an estimated 4321 articles (23.4%) were eligible for inclusion in this study—that is, as news reporting or analysis relevant to the country of publication and containing a direct focus on COVID-19 public health or policy information (SI Figs. S4 - S5 ). Articles with a direct focus on COVID-19 public health or policy information (to a small or large extent) could be coded for the scientific quality of the reporting of this information and its sensationalism.

For each randomly sampled day ( A ) and each news outlet ( B ), the total number of individual news records is shown, based on Factiva database searches for articles related to COVID-19 public health and policy information (Methods). News articles are partitioned across the following categories: articles eligible for inclusion in our study (eligible), articles not focusing on the newspaper’s country of publication (location out of scope), articles that are not original news reporting or analysis (opinion/editorial/letters), and articles that include COVID-19-relevant search terms, but do not include any direct focus on COVID-19 public health or policy information (no direct focus). Estimated totals for these categories are calculated using (i) the total number of Factiva returns and (ii) the rates at which articles were assigned to these categories during the eligibility screening process for each outlet and randomly sampled day (SI Fig. S4 ). On the stacked bars, percentages of articles falling into each category are specified for each day ( A ) and news outlet ( B ).

Content analysis of pandemic media coverage

We collected a manageable, well-defined random sample of 1331 news media articles satisfying our eligibility criteria (SI Fig. S4 ) for coding of scientific quality and sensationalism (SI Coding Tool and Dataset S1 ). Six questions each for scientific quality and for sensationalism were evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality or sensationalism, 1 corresponding to lowest scientific quality or sensationalism). Question categories included for scientific quality: applicability, opinion versus facts, validity, precision, context, and global assessment (i.e., an overall assessment of the article’s scientific quality); and for sensationalism: exposing, speculating, generalizing, warning, extolling, and global assessment (i.e., an overall assessment of the degree of sensationalism in the article) (SI Coding Tool). For this content analysis, interrater reliability was moderate to substantial for the summative “global” assessment of scientific quality and sensationalism (SI Table S2 ). Reliability was similarly high for specific scientific quality and sensationalism measures, with the exception of questions for which coded scores displayed restriction of range or unbalanced distributions (e.g., “generalizing” scores of mostly 1 and 2, rather than ranging from 1 through 5 with balance around 3; SI Coding Tool and Dataset S1 ) (Hallgren, 2012 ; Tran et al., 2020 ).

The scientific quality of pandemic media coverage

The scientific quality of news media articles differed among news outlets across the political spectrums of the respective countries (Fig. 3 ). Within each country, the overall scientific quality of news reporting and analysis was lowest on the populist-right of the political spectrum (mean summative “global” scientific quality of 2.58, n = 106 articles, for Toronto Sun ; 2.67, n = 115, for Daily Mail ; and 2.28, n = 118, for New York Post ; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for within-country pairwise comparisons except Daily Mail versus Times of London and Telegraph , SI Table S3 ). For these outlets, lower scientific quality was especially evident for validity, precision, and context as measures of scientific quality (e.g., articles reporting claims without fact checking, specificity, or background details) (Fig. 3 ).

Scores for six scientific quality questions (SI Coding Tool) are shown (mean, 95% confidence interval) for articles ( n  = 1331) communicating COVID-19 public health or policy information (Fig. 2 ): ( A ) applicability, ( B ) opinion versus facts, ( C ) validity, ( D ) precision, ( E ) context, and ( F ) global assessment (i.e., an overall assessment of the article’s scientific quality). Each question was evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality). Sampled articles were published between 1 March and 15 August 2020 (Methods). Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison test statistics are in SI Table S3 .

The sensationalism of pandemic media coverage

The sensationalism of news media articles was low overall for all news outlets, although somewhat greater for outlets on the left and middle of the political spectrum in Canada and the United States (Fig. 4F ). In both countries, news outlets at the populist-right combined low scientific quality with low sensationalism (Figs. 3 F and 4F ). In Canada, the overall sensationalism of news reporting and analysis was lowest for the Toronto Sun (mean summative “global” sensationalism of 1.77, n  = 106 articles; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons with Globe and Mail and National Post , SI Table S3 ). In the United States, overall sensationalism was lower in the Wall Street Journal (mean global sensationalism of 2.03, n  = 118 articles) and New York Post (mean of 2.13, n  = 118), as compared to the New York Times (mean of 2.40, n  = 120) and Washington Post (mean of 2.38, n  = 119; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons, SI Table S3 ). For these outlets, lower sensationalism was especially observed for exposing, speculating, and warning as measures of sensationalism (Fig. 4 ). In the United Kingdom, overall sensationalism did not vary across news outlets ( p  = 0.283 for Kruskal–Wallis, SI Table S3 ).

Scores for six sensationalism questions (SI Coding Tool) are shown (mean, 95% confidence interval) for articles ( n  = 1331) communicating COVID-19 public health or policy information (Fig. 2 ): ( A ) exposing, ( B ) speculating, ( C ) generalizing, ( D ) warning, ( E ) extolling, and ( F ) global assessment (i.e., an overall assessment of the degree of sensationalism in the article). Each question was evaluated on a scale from 1 to 5 (5 corresponding to highest sensationalism). Sampled articles were published between 1 March and 15 August 2020 (Methods). Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison test statistics are in SI Table S3 .

Syndicated versus original reporting

Across all outlets, the scientific quality of original reporting (mean global scientific quality of 2.93, n  = 1278 articles) was significantly higher than the scientific quality of syndicated articles (mean of 2.71, n  = 54; p  = 0.020, Kruskal–Wallis; SI Fig. S6 and Table S4 ). Additionally, the sensationalism of syndicated articles (mean global sensationalism of 1.82, n  = 54 articles) was significantly lower than the sensationalism of original reporting (mean of 2.14, n  = 1278; p  ≤ 0.001, Kruskal–Wallis; SI Fig. S6 and Table S4 ). The Toronto Sun published the highest proportion of syndicated news articles by far, with 34% of the paper’s 106 coded articles originating from syndicated sources. Other news outlets with more than 1% of coded articles drawing from syndicated sources included the Toronto Star (6% of articles) and the National Post (11%).

Neither scientific quality nor sensationalism varied substantially through time, with the exception of lower scientific quality on 3 July 2020 resulting from limited coverage of the healthcare sector that day (Fig. 5 , SI Fig. S7 ).

The topics of news media articles analyzed ( A ) over the timeframe of this study and ( B ) by news outlet are specified. Sampled articles were published on randomly sampled days between 1 March and 15 August 2020 (Methods). The topic of each article ( n  = 1331) was categorized by societal sectors (up to 2 selected per article) related to healthcare, leisure and entertainment, economics and commerce, government and politics, and other social services.

The topics of pandemic media coverage

News media articles were categorized based on the societal sectors (up to 2 per article) that were the primary focus of each article (Fig. 5 ). The sectors, related to healthcare, leisure and entertainment, economics and commerce, government and politics, and other social services, are listed in full in Figs. 5 and 6 . Although all analyzed articles contained information on the public-health effects of COVID-19 or measures to limit its spread (SI Fig. S4 ), topics of focus differed widely, for example including recreation, the arts, transportation, or daycare, not just medical facilities or vaccine research.

Scientific quality ( A ) and sensationalism ( B ) of news media articles are indicated by the topics of articles. Overall global assessment scores for scientific quality and sensationalism (SI Coding Tool) are shown (mean, 95% confidence interval) for articles communicating COVID-19 public health or policy information (Fig. 2 ). For each article, scientific quality and sensationalism were each evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality or to highest sensationalism). Sampled articles ( n  = 1331) were published between 1 March and 15 August 2020 (Methods). The topic of each article was categorized across the following societal sectors (up to two selected per article): healthcare and institutions; health-related medical and technology research; family, lifestyle, and social groups; professional or high-level sports; public parks and recreation; culture and the arts; private sector impacts and measures; employment impacts and benefits; macroeconomics and economy-wide fiscal or stimulus measures; politics and elections; law enforcement and court systems; foreign affairs and international aid; transportation, shipping, and border closures; education and daycare; public services; and energy and the environment.

The topics of news media articles corresponded to scientific quality and sensationalism of news reporting and analysis to some degree (Fig. 6 ). News media articles related to healthcare, health institutions, and health-related research were most common (Fig. 5 ), and they had significantly greater scientific quality compared to articles on other topics (mean global scientific quality of 3.23 for healthcare and institutions and 3.72 for health-related research; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons except with energy and the environment; Fig. 6A ). News media articles during the first four-week period studied, starting 1 March 2020, included the greatest focus (50.2% of coverage) on healthcare and related institutions and research (Fig. 5A ).

Sensationalism of articles related to politics and foreign affairs was greatest (mean global sensationalism of 2.53 for politics; and of 2.49 for foreign affairs; p  < 0.001 for Kruskal–Wallis, p  < 0.05 for pairwise comparisons of politics versus all sectors except foreign affairs, employment, and energy and the environment; Fig. 6B ). For example, sensational statements related to politics and foreign affairs could include exposing disinformation from political leaders or extolling political leaders for border closures as a pandemic or broader policy response. News outlets in the United States published the most articles related to politics and elections (63.8% of coverage across all outlets; Fig. 5B ).

Public-health policies consistently covered through time included measures related to social distancing, testing and tracing, and protective equipment and disinfection practices, while coverage of mask guidance and reopening policies increased over the course of the study (Fig. 7 ).

Sampled articles ( n  = 1331) were published on randomly sampled days between 1 March and 15 August 2020 (Methods). Public-health policies and measures in each article were coded under specific categories related to social distancing, testing and tracing, protective equipment and disinfection practices, reopening policies, vaccines and treatments, and more (all relevant categories selected for each article).

Managing the public health and societal risks of a pandemic requires iterative, informed decision-making by governments, individuals, and the private sector. News media play a central role in communicating public health and policy information, establishing accountability for decision-making, and shaping public perceptions through the number of news reports, their content, and their tone (Klemm et al., 2016 ; Reintjes et al., 2016 ). For news outlets spanning the political spectrum of three countries with contrasting public-health outcomes and policy responses (Fig. 1 ), based on a random sample of days, coverage related to COVID-19 increased substantially in March 2020 and declined gradually thereafter in May and June (Fig. 2 ), not rebounding even during the dramatic increase in U.S. COVID-19 cases in June and July (SI Figure S5 ). Understanding this news media reporting in the early stages of COVID-19 response provides important lessons for ensuring the accessibility of information in support of public health and gauging its degree of effectiveness in creating accountability for policy decisions.

News media reporting grappled with complications of scientific understanding and its uncertainties during the timeframe of our study, as assessed through our measures of validity, precision, and overall scientific quality. For example, the mechanisms of disease transmission, especially airborne transmission, were slow to be recognized, leading to dynamic adjustments of public-health guidance (e.g., for mask usage by the general public) (Zhang et al., 2020 ). Despite such uncertainties and frequent knowledge updates over time, the scientific quality of reporting was highest for the healthcare sector, also the most commonly occurring article topic (Fig. 6 ). The scientific quality of reporting overall did not improve as the pandemic proceeded and knowledge of COVID-19 increased, which may be attributed to shifts from healthcare to other topics of news media reporting (Fig. 5 and SI Fig. S7 ).

We did, however, identify major differences in the degree to which newspaper reporting of COVID-19 presented high-quality scientific information about the public-health effects of the coronavirus disease and measures to limit its spread. News media articles generally had moderate scientific quality overall (Fig. 3F ). Outlets on the populist-right of the political spectrum of each country, though, had significantly lower scientific quality in reporting related to COVID-19 (Fig. 3F ). Scientific quality was low especially for validity, precision, and context as measures of scientific quality, as well as for the distinction between opinion versus facts in some cases (e.g., articles reporting claims without fact checking, specificity, background details, or sourcing) (Fig. 3 ). These findings pertain to news reporting and analysis, rather than opinion pieces, editorials, or letters, which were excluded from the scope of news media articles we evaluated. The differences across outlets suggest that, in reading news reporting and analysis in different newspapers, readers access reporting of varying scientific quality related to the health risks and effectiveness of available measures to limit disease transmission.

Further, patterns of U.S. media reporting were correlated with failures of national leadership under the Trump Administration, and they may have both reflected and contributed to politicization of COVID-19 in the United States. During this study’s timeframe, the United States led the world in cases and deaths despite its pre-pandemic ranking as the country best equipped to manage a pandemic such as COVID-19 (Cameron et al., 2019 ). These public-health outcomes occurred against a backdrop of disinformation and failures of national leadership (Evanega et al., 2020 ; Ball and Maxmen, 2020 ; Holtz et al., 2020 ; Lincoln, 2020 ; Thorp, 2020 ). Lack of national leadership was observed in the relative dearth of national-level public-health policies and guidance (Fig. 1 ) and the divergence of subnational policy responses, correlated with partisan politics (SI Fig. S1 and Table S1 ). Elites and incumbent governments have outsize influence on public opinion and media coverage, which likely contributed to polarization and politicization of pandemic media coverage (Green et al., 2020 ; Hart et al., 2020 ). Linked to these trends, we observed higher sensationalism related to politics and elections topics and greater coverage of these sectors among U.S. newspapers (Figs. 5 – 6 ). Additionally, news outlets on the political left in the United States (i.e., New York Times , Washington Post ) published articles with more exposing and warning coverage, for example discussing disinformation on the part of government leaders and the risks of disease (Fig. 4 ). Although most Americans believe the media are fulfilling key roles during the pandemic, the majority of these individuals identify as Democrats, and Democrats trust many more new sources than individuals identifying as Republican (Jurkowitz et al., 2020 ; Gottfried et al., 2020 ).

In both Canada and the United States, low scientific quality was paired with lower-than-average sensationalism in news outlets on the populist-right (Figs. 3 F and 4F ). Sensationalism was low overall for all news outlets, but within Canada and the United States, it was lowest for the Toronto Sun and New York Post , as well as the Wall Street Journal . Although low sensationalism is generally considered beneficial, very low sensationalism combined with low scientific quality may have failed to alert readers to public-health risks and policy failures in some cases (e.g., per the measures of exposing and warning coverage in Fig. 4 ). Such trends also resulted, in part, from higher reliance on syndicated articles, especially in Canada, potentially related to structural and economic changes in news media (SI Fig. S6 ). Across the political spectrum, our results demonstrate that existing ideological perspectives may influence how information is used in reporting (Rosella et al., 2013 ). For example, news outlets at the populist-right in the United Kingdom and the United States may tend towards support of populist-right governments, demonstrating preference for those governments’ interpretation of the science, implemented policies, and use of science to justify choices made (Bennett et al., 2008 ; Grundmann and Stehr, 2012 ).

The studied news media outlets—traditional, national-level print media—have disproportionate influence on the content of other media platforms and on how that content is covered (Project for Excellence in Journalism, 2010 ; Denham, 2014 ). A better understanding of the effects of news media—or lack thereof—on public-health decision-making and public sentiment in the early stages of this pandemic can, for future pandemics or other public-health crises, increase public-health officials’ capacity to adapt communication strategies in disseminating guidance and coordinating responses of health system stakeholders (Laing, 2011 ; Rosella et al., 2013 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ; Pieri, 2019 ). Such understanding is crucial as the impacts of the policy actions themselves accumulate. The findings of this study point to complex interactions among scientific evidence on public-health risks and response measures, societal politicization of the science, and the scientific quality and sensationalism of media reporting. An inherent tension may exist: tendencies towards low sensationalism, especially combined with low scientific quality, may in some cases lead to characterization of public-health threats and policy failures as less extraordinary and relevant than they actually are.

Data availability

All data generated or analyzed during this study are included in this published article and its Supplementary Information .

Adeel AB, Catalano M, Catalano O, et al (2020) COVID-19 policy response and the rise of the sub-national governments. Can Public Policy. https://doi.org/10.3138/cpp.2020-101

Anderson B, Coletto D (2017) Canadian news media and “fake news” under a microscope. In: Abacus Data. https://abacusdata.ca/canadian-news-media-and-fake-news-under-a-microscope/ . Accessed 25 Aug 2020

Antoine J-Y, Villaneau J, Lefeuvre A (2014) Weighted Krippendorff’s alpha is a more reliable metrics for multi- coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation. In: Proceedings of the 14th conference of the European chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Gotenborg, Sweden, pp. 550–559

Austen I (2020) Canada’s largest newspaper changes hands amid vow to keep liberal voice. N. Y. Times

Ball P, Maxmen A (2020) The epic battle against coronavirus misinformation and conspiracy theories. Nature 581:371–374. https://doi.org/10.1038/d41586-020-01452-z

Article   ADS   CAS   PubMed   Google Scholar  

Bennett WL, Lawrence RG, Livingston S (2008) When the press fails: political power and the news media from Iraq to Katrina. University of Chicago Press

Cameron EE, Nuzzo JB, Bell JA (2019) GHS index: global health security index: building collective action and accountability. NTI and Johns Hopkins Bloomberg School of Public Health

CBC News (2020) Tracking the spread of coronavirus in canada and around the world. https://newsinteractives.cbc.ca/coronavirustracker/ . Accessed 25 Aug 2020

Centers for Disease Control and Prevention (CDC) (2020) CDC COVID Data Tracker. https://covid.cdc.gov/covid-data-tracker . Accessed 30 Aug 2020

Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46. https://doi.org/10.1177/001316446002000104

Article   Google Scholar  

Denham BE (2014) Intermedia attribute agenda setting in the New York Times: the case of animal abuse in U.S. horse racing. Journal Mass Commun Q 91:17–37. https://doi.org/10.1177/1077699013514415

Evanega S, Lynas M, Adams J, Smolenyak K (2020) Coronavirus misinformation: quantifying sources and themes in the COVID-19 ‘infodemic.’ Cornell Alliance for Science

Fleiss JL, Cohen J (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 33:613–619. https://doi.org/10.1177/001316447303300309

Gentzkow M, Shapiro JM (2010) What drives media slant? Evidence from U.S. daily newspapers. Econometrica 78:35–71. https://doi.org/10.3982/ECTA7195

Article   MathSciNet   Google Scholar  

Giraudoux P, Antonietti J-P, Beale C, et al (2018) pgirmess: spatial analysis and data mining for field ecologists. R package version 1.6.9 https://CRAN.R-project.org/package=pgirmess

Gottfried J, Walker M, Mitchell A (2020) Americans’ views of the news media during the Coronavirus outbreak. Pew Research Center

Government of Canada (2016) Canada’s health care system. https://www.canada.ca/en/health-canada/services/canada-health-care-system.html . Accessed 18 Oct 2020

Green J, Edgerton J, Naftel D et al. (2020) Elusive consensus: polarization in elite communication on the COVID-19 pandemic. Sci Adv 6:eabc2717. https://doi.org/10.1126/sciadv.abc2717

Article   ADS   PubMed   PubMed Central   Google Scholar  

Grossman G, Kim S, Rexer JM, Thirumurthy H (2020) Political partisanship influences behavioral responses to governors’ recommendations for COVID-19 prevention in the United States. Proc Natl Acad Sci USA 117:24144–24153. https://doi.org/10.1073/pnas.2007835117

Article   CAS   PubMed   PubMed Central   Google Scholar  

Grundmann R, Stehr N (2012) The power of scientific knowledge: from research to public policy. Cambridge University Press, Cambridge

Book   Google Scholar  

Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 8:23–34. https://doi.org/10.20982/tqmp.08.1.p023

Article   PubMed   PubMed Central   Google Scholar  

Hart PS, Chinn S, Soroka S (2020) Politicization and polarization in COVID-19 news coverage. Sci Commun 42:679–697. https://doi.org/10.1177/1075547020950735

Article   PubMed Central   Google Scholar  

Hoffman SJ, Justicz V (2016) Automatically quantifying the scientific quality and sensationalism of news records mentioning pandemics: validating a maximum entropy machine-learning model. J Clin Epidemiol 75:47–55. https://doi.org/10.1016/j.jclinepi.2015.12.010

Holtz D, Zhao M, Benzell SG et al. (2020) Interdependence and the cost of uncoordinated responses to COVID-19. Proc Natl Acad Sci USA 117:19837–19843. https://doi.org/10.1073/pnas.2009522117

Hönnige C, Nyhuis D, Meyer P et al. (2020) Dominating the debate: visibility bias and mentions of British MPs in newspaper reporting on Brexit. Polit Res Exch 2:1788955. https://doi.org/10.1080/2474736X.2020.1788955

Jurkowitz M, Mitchell A, Shearer E, Walker M (2020) U.S. media polarization and the 2020 election: a nation divided. Pew Research Center

Klemm C, Das E, Hartmann T (2016) Swine flu and hype: a systematic review of media dramatization of the H1N1 influenza pandemic. J Risk Res 19:1–20. https://doi.org/10.1080/13669877.2014.923029

Lacy S, Riffe D, Stoddard S et al. (2001) Sample size for newspaper content analysis in multi-year studies. J Mass Commun Q 78:836–45

Google Scholar  

Laing A (2011) The H1N1 crisis: roles played by government communicators, the public and the media. J Prof Commun 1:123–149. https://doi.org/10.15173/jpc.v1i1.88

Article   ADS   Google Scholar  

Lincoln M (2020) Study the role of hubris in nations’ COVID-19 response. Nature 585:325–325. https://doi.org/10.1038/d41586-020-02596-8

Mitchell A, Simmons K, Matsa KE, et al (2018) In Western Europe, public attitudes toward news media more divided by populist views than left-right ideology. Pew Research Center

Molek-Kozakowska K (2013) Towards a pragma-linguistic framework for the study of sensationalism in news headlines. Discourse Commun 7:173–197. https://doi.org/10.1177/1750481312471668

Oxman AD, Guyatt GH, Cook DJ et al. (1993) An index of scientific quality for health reports in the lay press. J Clin Epidemiol 46:987–1001. https://doi.org/10.1016/0895-4356(93)90166-x

Article   CAS   PubMed   Google Scholar  

Pieri E (2019) Media framing and the threat of global pandemics: the Ebola crisis in UK media and policy response. Sociol Res Online 24:73–92

Project for Excellence in Journalism (2010) How news happens: a study of the news ecosystem of one American city. Pew Research Center

Public Health England, NHSX (2020) Coronavirus (COVID-19) in the UK. https://coronavirus.data.gov.uk . Accessed 2 Oct 2020

Puglisi R, Snyder JM (2015) The balanced US press. J Eur Econ Assoc 13:240–264. https://doi.org/10.1111/jeea.12101

R Core Team (2020) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria

Reintjes R, Das E, Klemm C et al. (2016) “Pandemic public health paradox”: time series analysis of the 2009/10 influenza A / H1N1 epidemiology, media attention, risk perception and public reactions in 5 European countries. PLoS ONE 11:e0151258. https://doi.org/10.1371/journal.pone.0151258

Riffe D, Aust CF, Lacy SR (2016) The effectiveness of random, consecutive day and constructed week sampling in newspaper content analysis. Journal Q 70:133–139. https://doi.org/10.1177/107769909307000115

Rosella LC, Wilson K, Crowcroft NS et al. (2013) Pandemic H1N1 in Canada and the use of evidence in developing public health policies–A policy analysis. Soc Sci Med 83:1–9. https://doi.org/10.1016/j.socscimed.2013.02.009

Roser M, Ritchie H, Ortiz-Ospina E, Hasell J (2020) Coronavirus pandemic (COVID-19). Our World in Data. https://ourworldindata.org/coronavirus . Accessed 30 Aug 2020

Thorp HH (2020) Trump lied about science. Science 369:1409. https://doi.org/10.1126/science.abe7391

Tran D, Dolgun A, Demirhan H (2020) Weighted inter-rater agreement measures for ordinal outcomes. Commun Stat-Simul Comput 49:989–1003. https://doi.org/10.1080/03610918.2018.1490428

World Health Organization (WHO) (2020a) Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 . Accessed 7 Jun 2020

World Health Organization (WHO) (2020b) Timeline of WHO’s response to COVID-19. https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 5 Jun 2020

Zhang R, Li Y, Zhang AL et al. (2020) Identifying airborne transmission as the dominant route for the spread of COVID-19. Proc Natl Acad Sci USA 117:14857–14863. https://doi.org/10.1073/pnas.2009637117

Download references

Acknowledgements

S. Damouras provided advising on methods of statistical analysis, and J. Niemann formatted references. Funding for this work was provided by the University of Toronto Scarborough Department of Physical and Environmental Sciences and the University of Miami Rosenstiel School of Marine and Atmospheric Science.

Author information

Authors and affiliations.

Department of Environmental Science and Policy, Rosenstiel School of Marine and Atmospheric Science, University of Miami, Miami, FL, USA

Katharine J. Mach & Rosalind Donald

Leonard and Jayne Abess Center for Ecosystem Science and Policy, University of Miami, Coral Gables, FL, USA

Katharine J. Mach

Department of Physical and Environmental Sciences, University of Toronto, Toronto, ON, Canada

Raúl Salas Reyes, Brian Pentz, Clarissa A. Costa, Sandip G. Cruz, Kerronia E. Thomas & Nicole Klenk

Department of Geography and Planning, University of Toronto, Toronto, ON, Canada

Jennifer Taylor

Aspen Global Change Institute, Basalt, CO, USA

James C. Arnott

Earth and Environmental Sciences Area, Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Kripa Jagannathan

School for Environment and Sustainability, University of Michigan, Ann Arbor, MI, USA

Department of Civil and Environmental Engineering, University of Connecticut, Storrs, CT, USA

Christine J. Kirchhoff

Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

Laura C. Rosella

You can also search for this author in PubMed   Google Scholar

Contributions

All authors conceived the analysis. KJM, RSR, BP, JT, CAC, SGC, KET, and NK designed the methods of analysis with review by all authors. RSR, BP, JT, CAC, SGC, and KET collected data. KJM, RSR, BP, and JT performed analysis of data and developed visualizations of data. KJM, RSR, BP, JT, and NK drafted the manuscript with review and edits from all authors.

Corresponding author

Correspondence to Katharine J. Mach .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, si appendix dataset s1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Mach, K.J., Salas Reyes, R., Pentz, B. et al. News media coverage of COVID-19 public health and policy information. Humanit Soc Sci Commun 8 , 220 (2021). https://doi.org/10.1057/s41599-021-00900-z

Download citation

Received : 06 January 2021

Accepted : 15 September 2021

Published : 28 September 2021

DOI : https://doi.org/10.1057/s41599-021-00900-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Covid-19 pandemic and vaccination skepticism.

• Abdul Latif Anas
• Mashudu Salifu
• Hanan Lassen Zakaria

Human Arenas (2023)

Local TV News Coverage of Racial Disparities in COVID-19 During the First Wave of the Pandemic, March–June 2020

• Elizabeth K. Farkouh
• Jeff Niederdeppe

Race and Social Problems (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies