Introduction

COVID-19 is an infection caused by the SARS-CoV-2 coronavirus[1]. It was first identified in Wuhan, China, on December 31, 2019[2]. On March 11, 2020, the WHO characterized the COVID-19 outbreak as a pandemic[1]. In July 2020, more than fifteen million cases of contagion had been identified worldwide[3].

While the majority of cases result in mild symptoms, some might progress to pneumonia, acute respiratory distress syndrome, and death[4],[5],[6]. The case fatality rate reported across countries, settings, and age groups is highly variable, but it ranges from about 0.8% to 18%[7].

Multiple drugs have been proposed as a possible treatment for patients with moderate to severe COVID-19. Azithromycin, and other macrolides, have been suggested due to their alleged role in preventing bacterial superinfection and their immunomodulatory and anti-inflammatory effects[8],[9]. However, clinical studies evaluating the use of macrolides in the treatment of adult or pediatric patients with different respiratory infections, such as influenza or respiratory syncytial virus, have shown contradictory results[10],[11],[12],[13],[14],[15],[16],[17].

Despite these results, macrolides have been empirically prescribed in patients with pneumonia caused by novel coronaviruses such as SARS and MERS, and, more recently, SARS-CoV-2. Azithromycin attracted attention after the release of a non-randomized study—with considerable methodological limitations—and an observational study, claiming that hydroxychloroquine with azithromycin achieved a higher level of SARS-CoV-2 clearance in respiratory secretions[18],[19].

Using innovative and agile processes, taking advantage of technological tools, and resorting to several research groups' collective effort, this living, systematic review aims to provide a timely, rigorous, and continuously updated summary of the evidence available on patients with COVID-19.

Methods

Protocol and registration

This manuscript complies with the 'Preferred Reporting Items for Systematic reviews and Meta-Analyses' (PRISMA) guidelines[20]. Appendix 1 - PRISMA checklist.

A protocol stating the shared objectives and methodology of multiple evidence syntheses (systematic reviews and overviews of systematic reviews) to be conducted in parallel for different questions relevant to COVID-19 was published elsewhere[21]. This systematic review protocol was adapted to the specificities of the question[22] and submitted to PROSPERO CRD42020181032.

Search strategies

Electronic searches

Our literature search was devised by the team maintaining the L·OVE platform (https://app.iloveevidence.com), using the following approach:

1. Identification of terms relevant to the population and intervention components of the search strategy, using Word2vec technology[23] to the corpus of documents available in Epistemonikos Database.
2. Discussion of terms with content and methods experts to identify relevant, irrelevant and missing terms.
3. Creation of a sensitive boolean strategy encompassing all the relevant terms.
4. Iterative analysis of articles missed by the boolean strategy, and refinement of the strategy accordingly.

We conducted searches using the L·OVE (Living OVerview of Evidence) platform (https://app.iloveevidence.com) for COVID-19. This system maps PICO questions to a repository and is maintained through regular searches in 31 databases, preprint servers, trial registries, and websites relevant to COVID-19. The list of sources is regularly updated on our website. All the searches covered the period until August 6, 2020. No date, language, study design, publication status, or language restriction was applied to the searches in the Epistemonikos or the additional searches.

All the platform information comes from a repository developed and maintained by Epistemonikos Foundation through the screening of different sources relevant to COVID-19[24]. At the time of releasing this article, this repository included more than 65 000 articles pertinent to the  coronavirus disease, coming from the following databases, trial registries, preprint servers and websites relevant to COVID-19: Epistemonikos database, Pubmed/medline, EMBASE, CINAHL, PsycINFO, ICTRP Search Portal, Clinicaltrials.gov, ISRCTN registry, Chinese Clinical Trial Registry, IRCT - Iranian Registry of Clinical Trials, EU Clinical Trials Register: Clinical trials for covid-19, NIPH Clinical Trials Search (Japan) - Japan Primary Registries Network (JPRN) (JapicCTI, JMACCT CTR, jRCT, UMIN CTR), UMIN-CTR - UMIN Clinical Trials Registry, JRCT - Japan Registry of Clinical Trials, JAPIC Clinical Trials Information, Clinical Research Information Service (CRiS)- Republic of Korea, ANZCTR - Australian New Zealand Clinical Trials Registry , ReBec - Brazilian Clinical Trials Registry, CTRI - Clinical Trials Registry - India, RPCEC - Cuban Public Registry of Clinical Trials, DRKS - German Clinical Trials Register, LBCTR - Lebanese Clinical Trials Registry , TCTR - Thai Clinical Trials Registry, NTR - The Netherlands National Trial Register, PACTR - Pan African Clinical Trial Registry, REPEC - Peruvian Clinical Trial Registry, SLCTR - Sri Lanka, Clinical Trials Registry , medRxiv, bioRxiv and SSRN Preprints

The database[25] acts as a central repository. Only articles fulfilling Epistemonikos criteria are visible by users. The remaining articles are exclusively accessible for members of the COVID-19 L·OVE Working Group.

The following search strategy was used in Epistemonikos Database[25]. We adapted it to the syntax of other databases:

(coronavir* OR coronovirus* OR "corona virus" OR "virus corona" OR "corono virus" OR "virus corono" OR hcov* OR "covid-19" OR covid19* OR "covid 19" OR "2019-nCoV" OR cv19* OR "cv-19" OR "cv 19" OR "n-cov" OR ncov* OR "sars-cov-2" OR "sars-cov2" OR "SARS-Coronavirus-2" OR "SARS-Coronavirus2" OR (wuhan* AND (virus OR viruses OR viral)) OR (covid* AND (virus OR viruses OR viral)) OR "sars-cov" OR "sars cov" OR "sars-coronavirus" OR "severe acute respiratory syndrome" OR "mers-cov" OR "mers cov" OR "middle east respiratory syndrome" OR "middle-east respiratory syndrome" OR "covid-19-related" OR "SARS-CoV-2-related" OR "SARS-CoV2-related" OR "2019-nCoV-related" OR "cv-19-related" OR "n-cov-related") AND ((macrolide*) OR (fidaxomicin* OR clostomicin* OR lipiarm* OR "OPT-80" OR "OPT 80" OR OPT80* OR "PAR-01" OR "PAR 01" OR PAR01* OR "PAR-101" OR "PAR 101" OR PAR101* OR tiacumicin* OR Dificid* OR Dificlir* ) OR (azithromycin* OR Zithromax* OR Azithrocin*) OR (clarithromycin* OR Biaxin* ) OR (erythromycin* OR Eryc* OR Erythrocin* ) OR (josamycin* ) OR (solithromycin* OR "CEM-101" OR "CEM 101" OR CEM101* OR "OP-1068" OR "OP 1068" OR OP1068* OR Solithera*) OR (spiramycin* ) OR (troleandomycin* OR Triocetin* OR Tekmisin* ) OR (roxithromycin* ) OR (telithromycin* OR Ketek* ) OR (cethromycin* OR "ABT-773" OR "ABT 773" OR ABT773* OR Restanza*) OR (carrimycin*))

Eligibility criteria

Types of studies

We included randomized controlled trials. We excluded information from non-randomized studies, post-trial analyses, and studies evaluating animal models' effects or in vitro conditions.

Types of participants

We included trials assessing participants with confirmed COVID-19, as defined by the authors of the trials.

Whenever we found substantial clinical heterogeneity on how the condition was defined, we planned to explore it using a sensitivity analysis.

Type of interventions

The interventions of interest were macrolides (i.e., azithromycin, clarithromycin, erythromycin, carrimycine). We did not restrict our criteria to any dosage, duration, timing, or route of administration.

The comparison of interest was placebo (macrolides plus optimal treatment versus placebo plus optimal treatment) or no treatment (macrolides plus optimal treatment versus optimal treatment).
Trials assessing macrolides plus other drugs were eligible if the cointerventions are identical in both intervention and comparison groups. Trials evaluating macrolides in combination with other active drugs versus placebo or no treatment also were included.

Type of outcomes

We did not use the outcomes as an inclusion criterion during the selection process. Any article meeting all the criteria except for the outcome criterion was preliminarily included and assessed in full text.

We used the core outcome set COS-COVID[26], the existing guidelines and reviews, and the judgment of the authors of this review as an input for selecting the primary and secondary outcomes, as well as to decide upon inclusion. The review team revised this list of outcomes to incorporate ongoing efforts to define Core Outcomes Sets (e.g., COVID-19 Core Outcomes[27]).

The primary outcome was all-cause mortality. The secondary outcomes were mechanical ventilation, extracorporeal membrane oxygenation, length of hospital stay, respiratory failure, serious adverse events, and time to SARS-CoV-2 RT-PCR negativity. Other outcomes were acute respiratory distress syndrome and total adverse events.

We present primary and secondary outcomes in GRADE 'Summary of Findings' tables[28].

Selection of studies

The results of the literature search in all databases were automatically incorporated into the L·OVE platform (automated retrieval), where they were de-duplicated by an algorithm comparing unique identifiers (database ID, DOI, trial registry ID), and citation details (i.e., author names, journal, year of publication, volume, number, pages, article title and article abstract).

Two researchers independently screened the titles and abstracts yielded by the search against the inclusion criteria. We obtained the full reports for all titles that appeared to meet the inclusion criteria or required further analysis to decide their inclusion.

We recorded the reasons for excluding trials in any stage of the search. We outlined the study selection process in a PRISMA flow diagram adapted for this project.

Extraction and management of data

Using standardized forms, two reviewers independently extracted data from each included study. We collected the following information: study design, setting, participant characteristics (including disease severity and age) and study eligibility criteria; details about the administered intervention and comparison, including dose and therapeutic scheme, duration, timing (i.e., time after diagnosis), and route of administration; the outcomes assessed and the time they were measured; the source of funding of the study; the conflicts of interest disclosed by the investigators; and the risk of bias assessment for each study.

We resolved disagreements by discussion, and one referee adjudicated unresolved disagreements.

Risk of bias assessment

The risk of bias for each randomized trial was assessed using the 'risk of bias' tool (RoB 2.0: a revised tool to assess the risk of bias in randomized trials)[29]. Two reviewers independently assessed five domains of bias for each outcome result of all reported outcomes and time points. These five domains were: bias due to (1) the randomization process, (2) deviations from intended interventions (effects of assignment to interventions at baseline), (3) missing outcome data, (4) measurement of the outcome, and (5) selection of reported results. Answers to signaling questions and collectively supporting information were considered to lead to a domain‐level judgment in the form of 'Low risk of bias,' 'Some concerns,' or 'High risk of bias.' These domain‐level judgments informed an overall 'risk of bias' assessment for each result. Discrepancies between review authors were resolved by discussion to reach consensus. If necessary, a third review author was consulted to achieve a decision.

We considered the following factors as potential baseline confounders: age, comorbidities (e.g., cardiovascular disease; renal disease, eye disease, liver disease); co-interventions; and severity, as defined by the authors (i.e., respiratory failure vs. respiratory distress syndrome vs. ICU requirement).

Measures of treatment effect

We expressed the estimate of treatment effect of an intervention as risk ratios or odds ratios along with 95% confidence intervals for dichotomous outcomes. We used mean difference and standard deviation for continuous outcomes to summarize the data using a 95 percent confidence interval.

Strategy for data synthesis

If more than one trial was included, we planned to conduct a meta-analysis for studies clinically homogeneous using RevMan 5[30], using the inverse variance method with the random-effects model. For any outcomes where data were insufficient to calculate an effect estimate, we planned to present a narrative synthesis.

Subgroup and sensitivity analysis

We planned to perform subgroup analysis according to the definition of severe COVID-19 infection (i.e., respiratory failure vs. respiratory distress syndrome vs. ICU requirement). In case we identified significant differences between subgroups (test for interaction < 0.05), we considered reporting the results of individual subgroups separately.

We planned to perform sensitivity analysis, excluding studies with a high risk of bias. In cases where the primary analysis effect estimates and the sensitivity analysis effect estimates significantly differed, we considered presenting either the low risk of bias—adjusted sensitivity analysis estimates—or the primary analysis estimates but downgrading the certainty of the evidence because of risk of bias.

Assessment of certainty of evidence

We judged certainty of the evidence for all outcomes using the Grading of Recommendations Assessment, Development and Evaluation working group methodology (GRADE Working Group)[31], across the domains of risk of bias, consistency, directness, precision, and reporting bias. Certainty was adjudicated as high, moderate, low, or very low. For the main comparisons and outcomes, we prepared a Summary of Findings table[28],[32] and also an interactive Summary of Findings table (http://isof.epistemonikos.org/).

Living evidence synthesis

An artificial intelligence algorithm deployed in the Coronavirus/COVID-19 topic of the L·OVE platform will provide instant notification of articles with a high likelihood of being eligible. The authors will review them, decide upon inclusion, and update the review's living web version accordingly. We will consider resubmission to the journal if the direction of the effect on the critical outcomes changes or a substantial modification to the evidence's certainty.

This review is part of a larger project set up to produce multiple parallel systematic reviews relevant to COVID-19[21].

Results

Results of the search

The search in the L·OVE platform retrieved 424 references. We considered 260 as potentially eligible and retrieved and evaluated their full texts. A total of thirteen records were observational studies that assessed the intervention of interest. Only one study (a randomized clinical trial)[33] was eligible for inclusion. The study selection process is summarized in Figure 1 - PRISMA Flowchart.

Figure 1.

PRISMA Flowchart.

Full size

Description of the included studies

We included one randomized clinical trial[33], which included a total of 665 patients, but only 438 of them were randomized to the intervention and comparison groups of this systematic review's interest. In this trial, adult patients with suspected or confirmed COVID-19 were randomly assigned to one of three groups: standard care; hydroxychloroquine (400 milligrams twice daily for seven days) plus standard care; or azithromycin (400 milligrams twice daily for seven days) plus hydroxychloroquine (400 milligrams twice daily for seven days) plus standard care. For this systematic review, we extracted data from the latter two groups, which compares the isolated effect of azithromycin. Measured outcomes of our interest were: all-cause mortality, invasive mechanical ventilation, length of hospital stay, respiratory failure, serious adverse events, and total adverse events. Table 1 presents the inclusion criteria of the included study, and table 2 shows the main characteristics and baseline characteristics of the participants. We describe the details of the study in Appendix 2. 

Table 1.

Inclusion criteria of the study.

Full size

Table 2.

Main characteristics of the included study and baseline characteristics of the participants.

Full size

Ongoing studies

We identified 78 ongoing studies (71 randomized trials and seven non-randomized studies). See Appendix 2 - List of included, excluded, and ongoing studies.

Excluded studies

We excluded 165 studies that did not fulfill our eligibility criteria. A detailed list of excluded studies with reasons for exclusion is presented in Appendix 2 - List of included, excluded, and ongoing studies.

Risk of bias of the included study

The overall risk of bias was high, mainly due to deviations from the intended interventions. The other four domains (randomization process, missing outcomes, measurement of the outcome, and selection of reported results) were assessed as low risk of bias. Appendix 2 presents the main reasons for this assessment.

Effects of interventions

Of the outcomes of interest for this review, the included study did not report on "Extracorporeal membrane oxygenation" and "Time to SARS-CoV-2 RT-PCR negativity". The results for all other outcomes are presented in Table 3 and our interactive Summary of Findings table.

Table 3.

Summary of findings of macrolides for the treatment of COVID-19.

Full size

Primary outcome: All-cause mortality

The included study reported the effect of interventions on in-hospital death; we extracted this outcome as "All-cause mortality"[33]. The analysis showed a non-statistically significant difference in the risk of all-cause mortality between intervention groups (risk ratio: 0.57; 95% confidence interval: 0.19 to 1.66; 438 patients; 13 events) (Figure 2). 

Figure 2.

Relative risk for dichotomous outcomes for azithromycin plus hydroxychloroquine versus hydroxychloroquine.

Full size

Secondary outcomes:

1. Mechanical ventilation

The included study reported the effect of interventions on the need for invasive mechanical ventilation within fifteen days. Out of the 217 patients in the intervention group, twenty needed mechanical ventilation within fifteen days compared to the sixteen of 221 patients in the control group. Results did not show statistically significant differences between compared interventions on the need for mechanical ventilation (risk ratio: 1.27; 95% confidence interval: 0.68 to 2.39; 438 patients; 36 events). These results are shown in Figure 2.

2. Extracorporeal membrane oxygenation

The included study did not assess this outcome.

3. Length of hospital stay

The included study reported the length of hospital stay, and these results are shown in Figure 3. The mean duration of hospital stay for the 217 patients in the intervention group was 9.4 days (standard deviation of 7.8). In comparison, the mean duration of hospital stay for the 221 patients in the control group was 8.9 days (standard deviation of 6.2). The mean difference between groups showed very imprecise results from which to draw conclusions (mean difference 0.5; 95% confidence interval: -0.82 to 1.82; 438 patients).

Figure 3.

Mean difference for length of hospital stay for azithromycin plus hydroxychloroquine versus hydroxychloroquine.

Full size

4. Respiratory failure

The included study reported the need for high flow nasal cannula, non-invasive ventilation, and invasive mechanical ventilation within fifteen days. We extracted these outcomes as "Respiratory failure," and the results are shown in Figure 3. Out of 217 patients in the intervention group, 37 had respiratory failure, compared to 35 of 221 patients in the control group. The comparative analysis did not show statistically significant differences between compared interventions in the effect on respiratory failure (risk ratio: 1.08; 95% confidence interval: 0.71 to 1.64; 438 patients; 72 events).

5. Serious adverse events

The per-protocol analysis showed very imprecise results and did not show any trend regarding the risk of adverse events associated with the compared interventions (risk ratio: 2.08; 95% confidence interval: 0.41 to 10.61; 438 patients; 239 in the intervention group; 199 in the control group; seven events). This outcome is presented in Figure 3.

6. Time to SARS-CoV-2 RT-PCR negativity

The included study did not assess this outcome.

7. Total adverse events

The included study reported serious adverse events, and for any other adverse events, we extracted these two outcomes as "Total adverse events." The per-protocol analysis results did not show statistically significant differences between compared interventions in terms of adverse events (risk ratio: 1.19; 95% confidence interval: 0.94 to 1.52; 438 patients; 239 in the intervention group; 199 in the control group; 168 events).

Discussion

This living, systematic review included only one randomized controlled trial that evaluated using azithromycin associated with hydroxychloroquine compared to hydroxychloroquine alone for COVID-19[33]. Its results show that there is not enough evidence to conclude any difference between the intervention and control groups. All assessed outcomes had a wide confidence interval and were evaluated with small sample size, and therefore had a low or very low certainty of evidence.

The severity of some COVID-19 cases[34], added to the lack of a good treatment strategy and the production of vaccines still under development, has led to repurposing different drugs[35]; macrolides are some of them. The use of macrolides was proposed at the beginning of the pandemic due to its immunomodulatory, anti-inflammatory, and anti-viral effects shown in other viral diseases[36],[37],[38]. Gautret. et al.[18] proposed the association of azithromycin and hydroxychloroquine as a possible treatment strategy when they reported a 100% viral clearance in nasopharyngeal swabs in a total of six patients studied. As well as macrolides, hydroxychloroquine has its own immunomodulatory and anti-inflammatory effects[39]. To consider this association as a possible one, we must consider both drugs' adverse effects and their additive toxicity[35]. Both drugs are known for prolonging the QTc interval[40],[41],[42],[43], and it is likely to be suspected that this effect could increase when prescribing the drugs simultaneously[44],[39]. Additionally, we must consider that patients with severe COVID-19 are elderly and have pre-existing comorbidities, so adding a potentially risky combination of drugs may represent a challenge for the patient's health. The results of the included study in this living, systematic review showed a higher frequency of QTc interval increase in the group treated with hydroxychloroquine and azithromycin than the group that received hydroxychloroquine alone[33]. The group treated with azithromycin and hydroxychloroquine also had a higher frequency of any adverse events and elevation of liver enzyme levels[33], suggesting that additive toxicity between these drugs exists.

A still-increasing amount of evidence studying this intervention has been developed. Several observational studies have assessed azithromycin and hydroxychloroquine compared to hydroxychloroquine alone or azithromycin alone compared to standard treatment[45],[46],[47],[48],[49],[50],[51],[52],[53],[54],[55],[56]. The 78 ongoing studies indicated in this review are studying the same intervention as well. Systematic reviews have also been published evaluating the combination of azithromycin with hydroxychloroquine. Turgeon et al. searched the literature evaluating pharmacological properties and toxicity of six different drugs repurposed for COVID-19—including azithromycin—and their results showed that some cases report torsades de pointes after the administration of the macrolide[57]. Yang et al. concluded that the combination of hydroxychloroquine and azithromycin for COVID-19 might be effective because it showed synergic effects[58]. Gbinigie and Frie's rapid review concluded that there is limited evidence to confirm a synergic effect between azithromycin and hydroxychloroquine and that this evidence combined with its high risk of bias does not permit to support the use of azithromycin alone to treat COVID-19 (unless it is used in a trial or to treat a bacterial super-infection)[59]. Recently, a living, systematic review, and network meta-analysis about drug treatments for COVID-19 has been published[60]. However, Siemieniuk et al. did not include the randomized clinical trial included for this review due to the latter's recent publication[33],[60]. The reviews mentioned differ in methodology and conclusions. Our review only assesses one randomized controlled study, so it seems that the question is yet to be answered (by this review's update or other reviews that include the upcoming studies).

What draws our attention—and we cannot look away—is that one single methodologically-poor study initiated a waterfall of studies on the intervention of interest to this review. With "100% of viral clearance in nasopharyngeal swabs", Gautret et al.[18] proposed the intervention as the "heroin of the pandemic," and it was very far from being so. This then led to significant expenditures in different healthcare systems worldwide to treat their patients with the miraculous combination. In a chaotic healthcare system era, where everything is being reinvented, and telemedicine is the safest option to practice medicine[61], it appears peculiar to invest money in therapies that have not yet been proven useful.

This living, systematic review is not exempt from limitations, principally because it only assesses one randomized controlled trial. One of the reported outcomes in this review is "Respiratory failure," which included the patients in mechanical ventilation reported in the study. This must not be understood as a double report of outcome, but as an impossibility to report "Respiratory failure" without them because, obviously, patients in mechanical ventilation are in respiratory failure. The same outcome may be overestimating the number of patients in respiratory failure because the study does not clarify if a patient received more than one ventilatory assistance strategy. And the third limitation is that the results presented here only reflect the ones given by the first randomized clinical trial published assessing this intervention. There are no significant conclusions based on the results provided by one study because we lack a larger sample size. This limitation will be palliated in the future when the ongoing studies shown in this review become published. And this is where this review's strength appears: as it has a living method, it will be updated frequently to find new evidence when it becomes available. This type of review is particularly useful in this type of situation: the clinical question is relevant (pandemic with no clear treatment strategy nor vaccines), existing certainty of the evidence is low or very low (multiple observational studies, only one randomized controlled trial) and the information available will surely increase (over 70 ongoing studies evaluating the intervention presented in this review)[62]. The living method ensures future versions of this review that will include the evidence as it becomes available, so their results and conclusions will be very useful for future research and clinical practice.

This review is part of a larger project set up to put such an approach into practice. This project aims to produce multiple parallel living systematic reviews relevant to COVID-19 following the higher quality standards in evidence synthesis production[21]. We believe that our methods are well suited to handle the abundance of evidence that is to come, including evidence on macrolides' role for COVID-19. We have identified multiple ongoing studies addressing this question, including 71 randomized trials, which will provide valuable evidence to inform researchers and decision-makers soon.

Conclusions

Multiple drugs have been proposed as possible therapies for patients with moderate to severe COVID-19. Azithromycin, and other macrolides, have been suggested as a potential treatment due to their alleged role in preventing bacterial superinfection and their immunomodulatory and anti-inflammatory effects. Macrolides in the management of patients with COVID 19 showed no beneficial effects compared to standard of care. The evidence for all outcomes is inconclusive. Larger trials are needed to determine macrolides' effect on pulmonary and other outcomes in COVID 19 patients.

During the COVID-19 pandemic, we will maintain a living, web-based, openly available version of this review. We will re-submit the review every time the conclusions change or whenever there are substantial updates. Our systematic review aims to provide a high-quality, up-to-date synthesis of the evidence useful for clinicians and other decision-makers.

Differences between protocol and review

In this review, some methods were not implemented. We did not implement either the search in other sources specified in the original protocol[22] or the screening in Collaboratron™ because we considered L·OVE platform as a comprehensive tool that gathers both processes, including diverse databases and registries, as we specified in our methods. We neither needed to express continuous outcomes as a standardized mean difference because we only included one randomized clinical trial in our review. No minimally important difference (MID) was known, so we could not express continuous outcomes as MID units.