Introduction

COVID-19 is an infection caused by the SARS-CoV-2 coronavirus[1]. It was first identified in Wuhan, China, on December 31, 2019[2]. By May 24, 2020, the number of confirmed COVID-19 cases had reached 5 400 608, with 344 077 confirmed deaths[3]. On March 11, 2020, the WHO characterized the COVID-19 outbreak as a pandemic[1].

While the majority of cases result in mild symptoms, some might progress to pneumonia, acute respiratory distress syndrome, and death[4],[5],[6]. The case fatality rate reported across countries, settings, and age groups is highly variable, but it ranges from about 0.5% to 10%[7]. In some centers, it has been reported to be higher than 10% in hospitalized patients[8].

Lopinavir/ritonavir is a fixed-dose combination antiviral widely used for HIV infection, and it has been suggested as a possible treatment in the context of the COVID-19 pandemic in 2019/2020. This antiviral inhibits the HIV protease enzyme, forming an inhibitor-enzyme complex, thereby preventing cleavage of the gag-pol polyproteins.

The information about its antiviral properties against coronavirus comes from its use in the previous SARS-CoV and MERS-CoV pandemics. In vitro studies demonstrated an antiviral activity of lopinavir/ritonavir against SARS-CoV[9], and clinical studies reported a reduction in the intubation rate, steroid requirements and mortality with lopinavir/ritonavir in SARS patients[10]. However, several systematic reviews evaluating the role of lopinavir/ritonavir concluded it was not possible to establish its efficacy, based on the results of non-randomized studies[11],[12],[13].

With innovative and agile processes, technological tools, and the collective effort of several research groups, this living systematic review aims to provide a timely, rigorous, and continuously updated summary of the evidence available on the role of lopinavir/ritonavir in the treatment of patients with COVID-19.

Methods

This manuscript complies with the ‘Preferred Reporting Items for Systematic reviews and Meta-Analyses’ (PRISMA) guidelines for reporting systematic reviews and meta-analyses[14] (see Appendix 1 - PRISMA Checklist).

A protocol stating the shared objectives and methodology of multiple evidence syntheses (systematic reviews and overviews of systematic reviews) to be conducted in parallel for different questions relevant to COVID-19 was published elsewhere[15]. The review was registered in PROSPERO with the number CRD42020179212, and a detailed protocol was uploaded to a preprint server[16].

Search strategies 

Electronic searches

Our literature search was devised by the team maintaining the L·OVE platform, using the following approach:

1. Identification of terms relevant to the population and intervention components of the search strategy, using Word2vec technology[17] to the corpus of documents available in Epistemonikos Database;
2. Discussion of terms with content and methods experts to identify relevant, irrelevant and missing terms;
3. Creation of a sensitive boolean strategy encompassing all the relevant terms;
4. Iterative analysis of articles missed by the boolean strategy, and refinement of the strategy accordingly.

Our primary search source was the Epistemonikos database, a comprehensive database of systematic reviews and other types of evidence[18] that we have supplemented with information from 33 sources relevant to COVID-19. The list of sources that have been added to Epistemonikos is continuously expanded. This list of sources regularly screened by Epistemonikos for COVID-19 is updated regularly on our website[19].

We conducted additional searches using highly sensitive searches in PubMed/MEDLINE, the Cochrane Central Register of Controlled Trials (CENTRAL), and Embase. The searches in Epistemonikos are continuously updated[19] and were last checked for this review the day before submission to the journal (May 24, 2020). The additional searches covered the period from the inception date of each database until May 2, 2020. No study design, publication status, or language restriction was applied to the searches in Epistemonikos or the additional electronic searches.

The following strategy was used to search in Epistemonikos Database. We adapted it to the syntax of other databases (see Appendix 2 - Search strategies):

(coronavir* OR coronovirus* OR "corona virus" OR "virus corona" OR "corono virus" OR "virus corono" OR hcov* OR "covid-19" OR covid19* OR "covid 19" OR "2019-nCoV" OR cv19* OR "cv-19" OR "cv 19" OR "n-cov" OR ncov* OR "sars-cov-2" OR "sars-cov2" OR (wuhan* AND (virus OR viruses OR viral)) OR (covid* AND (virus OR viruses OR viral)) OR "sars-cov" OR "sars cov" OR "sars-coronavirus" OR "severe acute respiratory syndrome" OR "mers-cov" OR "mers cov" OR "middle east respiratory syndrome" OR "middle-east respiratory syndrome" OR "covid-19-related" OR "SARS-CoV-2-related" OR "SARS-CoV2-related" OR "2019-nCoV-related" OR "cv-19-related" OR "n-cov-related") AND (lopinavir* OR "ABT-378" OR "ABT 378" OR ABT378* OR Kaletra* OR ritonavir* OR Norvir)

Other sources

To identify articles that might have been missed in the electronic searches, we proceeded as follows:

1. We screened the reference lists of other systematic reviews.
2. We scanned the reference lists of selected guidelines, narrative reviews, and other documents.
3. We reviewed websites specialized in COVID-19 (see Appendix 2).
4. We emailed the contact authors of all the included studies to ask for additional publications or data on their studies and other studies on the topic. 
5. We conducted cross-citation searches in Google Scholar and Microsoft Academic, using each included study as the index reference.
6. We reviewed the reference list of each included study.

Eligibility criteria

Types of studies

This living review preferentially includes randomized trials. Non-randomized comparative studies are included, but their information is only used when there is no direct evidence from randomized trials, or the certainty of the evidence for the critical outcomes resulting from the randomized trials is graded as low-or very low[20]. We excluded studies evaluating the effects on animal models or in vitro conditions.

Types of participants

We included trials assessing participants with COVID-19, as defined by the authors of the trials. 

When we did not find direct evidence from randomized trials, or if the evidence from randomized trials provides low- or very low-certainty evidence for critical outcomes, we considered eligible randomized trials evaluating lopinavir/ritonavir-based in other coronavirus infections, such as MERS-CoV or SARS-CoV infections[20].

Type of interventions

The intervention of interest is the combination of lopinavir/ritonavir or lopinavir alone. We did not restrict our criteria to any dosage, duration, timing, or route of administration. The comparison of interest is placebo (lopinavir ± ritonavir plus standard treatment versus placebo plus standard treatment) or no treatment (lopinavir ± ritonavir plus optimal treatment versus standard treatment).

Trials assessing lopinavir ± ritonavir plus other drugs are eligible if the cointerventions are identical in both intervention and comparison groups. Trials evaluating lopinavir ± ritonavir in combination with other active drugs versus placebo or no treatment were also included.

Type of outcomes

We did not use the outcomes as inclusion criteria during the selection process. Any article meeting all the criteria except for the outcome criterion was preliminarily included and assessed in full text. 

We used the core outcome set COS-COVID[21], the existing guidelines and reviews, and the judgment of the authors of this review as an input for selecting the primary and secondary outcomes, as well as to decide upon inclusion. The review team regularly revised this list of outcomes, in order to incorporate ongoing efforts to define Core Outcomes Sets (e.g., COVID-19 Core Outcomes[22].

The primary outcome was all-cause mortality. The secondary outcomes were mechanical ventilation, extracorporeal membrane oxygenation, length of hospital stay, respiratory failure, serious adverse events, time to SARS-CoV-2 RT-PCR negativity, acute respiratory distress syndrome, and total adverse events. The primary and secondary outcomes are presented in the GRADE ‘Summary of Findings’ tables. A table with all the outcomes is presented as an appendix[23]. 

Selection of studies

The results of the literature search in Epistemonikos are automatically incorporated into the L·OVE platform (automated retrieval). There, they are de-duplicated by an algorithm that compares unique identifiers (database ID, DOI, trial registry ID), and citation details (i.e., author names, journal, year of publication, volume, number, pages, article title, and article abstract). The additional searches are uploaded to the screening software Collaboratron™[24].

In both L·OVE platform and Collaboratron™, two researchers independently screened the titles and abstracts yielded by the search against the inclusion criteria. We obtained the full reports for all titles that appeared to meet the inclusion criteria or required further analysis and then decided about their inclusion.

We recorded the reasons for excluding trials in any stage of the search and outline the study selection process in a PRISMA flow diagram, which we adapted for this project.

Extraction and management of data

Using standardized forms, two reviewers independently extracted data from each included and ongoing study. We collected the following information: study design, setting, baseline participant characteristics (including disease severity, age, gender, comorbidities, time from onset to treatment, amount of supplemental oxygen, patients receiving mechanical ventilation) and study eligibility criteria; details about the administered intervention and comparison, including, dose, duration, and timing (i.e., the time after diagnosis); the outcomes assessed and the time they were measured; the source of funding of the study and the conflicts of interest disclosed by the investigators; the risk of bias assessment for each study. We resolved disagreements by discussion, and one arbiter adjudicated unresolved disagreements.

Risk of bias assessment

The risk of bias for each randomized trial was assessed by using the 'Risk of bias' tool (RoB 2.0: a revised tool to assess the risk of bias in randomized trials)[25]. We considered the effect of the assignment to the intervention for this review. Two reviewers independently assessed five domains of bias for each outcome result of all reported outcomes and time points. These five domains regard bias due to (1) the randomization process, (2) deviations from intended interventions (effects of assignment to interventions at baseline), (3) missing outcome data, (4) measurement of the outcome, and (5) selection of reported results. Answers to signaling questions and collectively supporting information lead to a domain‐level judgment in the form of 'Low risk of bias,' 'Some concerns,' or 'High risk of bias.' These domain‐level judgments inform an overall 'risk of bias' judgment for each result. Discrepancies between review authors are resolved by discussion to reach consensus. If necessary, a third review author was consulted to reach a decision.

We assessed the risk of bias of other study designs with the ROBINS‐I tool (ROBINS-I: Risk Of Bias In Non-randomised Studies of Interventions)[26]. We addressed the following domains: bias due to confounding, bias in the selection of participants into the study, bias in classification of interventions, bias due to deviations from intended interventions (effect of assignment to intervention), bias due to missing data, bias in the measurement of outcomes and bias in the selection of the reported result. We judged each domain as low risk, moderate risk, serious risk, critical risk, or no information, and evaluate individual bias items as described in ROBINS-I guidance. We did not consider time‐varying confounding, as these confounders are not relevant in this setting[26]. We consider the following factors as potential baseline confounders: age, comorbidities (e.g., cardiovascular disease, renal disease, eye disease, liver disease, metabolic comorbidities, asthma, COPD, smokers), co-interventions, and severity, as defined by the authors (i.e., respiratory failure versus respiratory distress syndrome versus ICU requirement).

Measures of treatment effect

For dichotomous outcomes, we expressed the estimate of the treatment effect of an intervention as risk ratios or odds ratios along with 95% confidence intervals. For continuous outcomes, we used the mean difference and standard deviation to summarise the data using a 95% confidence interval. Whenever continuous outcomes are measured using different scales, the treatment effect is expressed as a standardized mean difference with 95% confidence interval. When possible, we multiplied the standardized mean difference by a standard deviation representative from the pooled studies, for example, the standard deviation from a well-known scale used by several of the studies included in the analysis on which the result is based. In cases where the minimally important difference is known, we present continuous outcomes as minimally important difference units or inform the results as the difference in the proportion of patients achieving a minimal important effect between intervention and control[27]. Then, these results are displayed on the 'Summary of Findings Table' as a mean difference[27].

Strategy for data synthesis

The search results and the study selection are presented with flow charts and tables, according to recommendations of the PRISMA statement[14]. For any outcomes where it is not possible to calculate an effect estimate, a narrative synthesis is presented, describing the studies in terms of the direction and the size of effects, and any available measure of precision. For the main comparisons and outcomes, we prepare GRADE Summary of Findings tables[27],[28], and also interactive Summary of Finding tables. A Summary of Findings table with all the comparisons and outcomes is included as an appendix. For any outcomes where data is available from more than one trial, we conducted a formal quantitative synthesis (meta-analysis) for studies clinically homogeneous using RevMan 5[29] or other software, using the inverse variance method with the random-effects model. We assessed inconsistency by visual inspection of the forest plots and using the I² index.

Subgroup and sensitivity analysis

We performed subgroup analysis according to the definition of severe COVID-19 infection (i.e., respiratory failure versus respiratory distress syndrome versus intensive care unit requirement). In case we identify significant differences between subgroups (test for interaction < 0.05), we report the results of individual subgroups separately.

We performed a sensitivity analysis excluding studies with a high risk of bias, and if non-randomized studies are used, excluding studies that do not report adjusted estimates. In cases where the primary analysis effect estimates and the sensitivity analysis effect estimates significantly differ, we either present the low risk of bias-adjusted sensitivity analysis estimates or the primary analysis estimates but downgrading the certainty of the evidence because of risk of bias.

Assessment of certainty of the evidence

The certainty of the evidence for all outcomes is appraised using the Grading of Recommendations Assessment, Development, and Evaluation working group methodology (GRADE Working Group)[30], across the domains of risk of bias, consistency, directness, precision and reporting bias. Certainty was adjudicated as high, moderate, low, or very low. For the main comparisons and outcomes, we prepared Summary of Findings tables[27],[28], and also interactive Summary of Findings tables. A Summary of Finding table with all the comparisons and outcomes is included as an appendix.

Living evidence synthesis

An artificial intelligence algorithm deployed in the Coronavirus/COVID-19 topic of the L·OVE platform provides instant notification of articles with a high likelihood of being eligible. The authors reviewed them, decided upon inclusion, and updated the living web version of the review accordingly. We expect to resubmit to the journal any time there is a change in the direction of the effect on the critical outcomes or a substantial modification to the certainty of the evidence. This review is part of a larger project set up to produce multiple parallel systematic reviews relevant to COVID-19[15].

Results

Identification of studies 

We used a repository that includes searches in 35 trial registries, pre-print servers, and websites specialized in COVID-19. We also conducted additional searches in three electronic databases and scanned the references of multiple guidelines, reviews, and other documents.

The search in the L·OVE platform retrieved 292 records, and the additional searches retrieved 576 records (total records screened = 862). We considered 272 references as potentially eligible and retrieved and evaluated their full texts. Twelve studies were finally included[31],[32], including two randomized trials that provided direct evidence for the critical outcomes, so indirect evidence from other coronaviruses and non-randomized studies in COVID-19 were not considered for the estimation of the effects. We identified 61 ongoing studies (47 randomized trials and 14 non-randomized studies). The list of included, excluded, and ongoing studies are presented in Appendix 3. Figure 1 shows the flow diagram of the study selection process.

Figure 1.

PRISMA Flowchart (prepared by the authors from the study data).

Full size

Description of the included studies

Two trials (n=250) evaluated lopinavir/ritonavir in addition to standard care versus standard care alone in adult inpatients from China[31],[32]. Both trials included patients with radiologically confirmed pneumonia. One trial had additional criteria of severity[31], and the other only required the presence of symptoms[32]. The inclusion criteria used in the selected studies are shown in Table 1. Table 2 presents the characteristics of the intervention. Table 3 presents the baseline characteristics of the participants.

Table 1.

Inclusion criteria of the studies.

Full size

Table 2.

Characteristics of the intervention.

Full size

Table 3.

Baseline characteristics of the participants.*

Full size

Risk of bias in the included studies

Both included trials had issues with blinding, so they were rated as 'some concerns' for all outcomes. The first study was not blinded for participants or investigators[31]. The second study was not blinded, except for outcome assessors[32]. Appendix 3 summarises the risk of bias assessments.

Efficacy of lopinavir/ritonavir for the treatment of COVID-19

The main results are summarised in Table 4 - Summary of Findings. An evidence profile for all the outcomes is presented in Appendix 4.

Table 4.

Summary of findings of lopinavir/ritonavir in patients with COVID-19 (GRADE SoF table).

Full size

All-cause mortality 

Mortality was reported in both studies[31],[32] at 21 and 28 days. There were no deaths in one trial[32]. In the other trial, 25 (25%) of 100 patients assigned to the control group died compared with 19 (19.2%) of 99 patients assigned to lopinavir/ritonavir (relative risk 0.77, 95% confidence interval 0.45 to 1.30). The certainty of the evidence was judged as low because of the risk of bias in the study and the imprecision of the result.

In absolute terms, the magnitude of the reduction in mortality would be trivial in non-severe patients (2 less per 1000, 95% confidence interval 5.5 less to 3 more), moderate in severe non-critical patients (21 less per 1000, 95% confidence interval 49.5 less to 27 more) and large in critical patients (69 less per 1000, 95% confidence interval 165 less to 90 more).

Figure 2.

The relative risk for all-cause mortality for lopinavir/ritonavir versus standard care (prepared by the authors from the study data).

Full size

Invasive mechanical ventilation

The need for invasive mechanical ventilation was reported in only one trial at 28 days[31]. Fourteen (14.1%) of 99 patients assigned to lopinavir/ritonavir required mechanical ventilation compared with 18 (18%) of 100 patients assigned to the control group (relative risk 0.79, 95% confidence interval 0.41 to 1.49, absolute reduction 38 less per 1000, 95% confidence interval 106 less to 88 more). The certainty of the evidence was judged as low because of the risk of bias in the study and the imprecision of the result.

Figure 3.

The relative risk of requiring invasive mechanical ventilation for lopinavir/ritonavir versus standard care (prepared by the authors from the study data).

Full size

Extracorporeal membrane oxygenation

The risk of requiring extracorporeal membrane oxygenation was reported in one trial[31]. Two (2.0%) of 99 patients assigned to lopinavir/ritonavir required ECMO and 2 (2.0%) of 100 patients assigned to the control group (relative risk,0.79; 95% confidence interval 0.41 to 1.49, absolute reduction 0 per 1000, 95% confidence interval 17 less to 121 more). The certainty of the evidence was judged as low because of the risk of bias in the study and the imprecision of the result.

Figure 4.

The relative risk of requiring extracorporeal membrane oxygenation for lopinavir/ritonavir versus standard care. (prepared by the authors from the study data).

Full size

Length of hospital stay 

The length of hospital stay was reported in one study on day 28[31]. The mean duration of the hospital stay in 100 patients assigned to the control group was 16 days (interquartile range 13 to 18) compared to 14 days (interquartile range 12 to 17) of 99 patients assigned to lopinavir/ritonavir (mean difference 1 day, 95% confidece interval 0 to 3). The certainty of the evidence was judged as low because of the risk of bias in the study and the imprecision of the result.

Respiratory failure

None of the studies reported the number of patients that suffered respiratory failure. One study reported the number of patients who suffered respiratory failure and acute respiratory distress syndrome as a combined outcome on day 28[31]. Twenty-seven (27.3%) of the 99 patients assigned to the control group had respiratory failure or acute respiratory distress syndrome compared with 15 (19.2%) of 95 patients assigned to lopinavir/ritonavir (relative risk 0.56, 95% confidence interval 0.32 to 0.99). The certainty of the evidence was judged as low because of the risk of bias in the study and the imprecision of the result.

Time to SARS-CoV-2 RT-PCR negativity 

Time to the positive-to-negative conversion of SARS-CoV-2 nucleic acid was reported in one study during the 21-day follow-up period. The mean was 9.0 days (standard deviation 5.0) in 34 patients assigned to lopinavir/ritonavir and 9.3 days (standard deviation 5.2) in 17 patients assigned to the control group (difference 0.3 days less, 95% confidence interval 3.29 less to 2.69 more ). The certainty of the evidence was judged as very low because of the risk of bias in the study and very serious imprecision of the result.

Figure 5.

Time to SARS-CoV-2 RT-PCR negativity for lopinavir/ritonavir versus standard care (prepared by the authors from the study data).

Full size

Serious adverse events

Serious adverse events were reported in both studies[31],[32]. One study reported that 32 (32.3%) of the 99 patients assigned to the control group had serious adverse events compared with 19 (20%) of 95 patients assigned to lopinavir/ritonavir[31]. Another study reported that none of the 34 patients assigned to the control group had serious adverse events compared to one (2.9%) of 34 patients assigned to lopinavir/ritonavir[32]. The relative risk was 0.63 (95% confidence interval 0.39 to 1.03, an absolute reduction of 102 less per 1000, 95% confidence interval 168 less to 8 more). The certainty of the evidence was judged as low because of the risk of bias of the study and the imprecision of the results.

Figure 6.

Relative risk of serious adverse events for lopinavir/ritonavir versus standard care.

Full size

Total adverse events

Total adverse events were reported in both studies[31],[32]. One study reported that 49 (49.5%) of the 99 patients assigned to the control group had serious adverse events compared with 46 (48.4%) of 95 patients assigned to lopinavir/ritonavir[31]. Another study reported that none of the 17 patients assigned to the control group had serious adverse events compared to 12 (35.3%) of 34 patients assigned to lopinavir/ritonavir[32]. The relative risk was 2.42 (95% confidence interval 0.22 to 26.91, absolute reduction 600 more per 1000, 95% confidence interval 329 less to 1,000 more), and the certainty of the evidence was judged as very low because of risk of bias, imprecision, and inconsistency.

Considering the limited information from the trials for this outcome, we incorporated indirect evidence about adverse effects in other populations to the Summary of Findings Table.

Figure 7.

Relative risk for total adverse events for lopinavir/ritonavir versus standard care.

Full size

Discussion

We performed a comprehensive search of the literature in order to identify and summarize the evidence evaluating the effect of lopinavir/ritonavir in patients with COVID-19. Anticipating the lack of the available evidence, we also searched for non-randomized, comparative studies in COVID-19, and for randomized trials evaluating other coronavirus infections, such as MERS-CoV and SARS-CoV. We found two randomized trials evaluating lopinavir/ritonavir in addition to standard care versus standard care alone in 250 adult inpatients from China[31],[32].

Low certainty evidence shows lopinavir/ritonavir may reduce mortality, but as the magnitude of the reduction varies across different risk groups, any treatment decision should also vary. The evidence also shows that lopinavir/ritonavir may reduce the risk of developing respiratory failure, acute respiratory distress syndrome, or requiring invasive mechanical ventilation, but the certainty of this evidence is low. Then again, lopinavir/ritonavir may not lead to a substantial increase in the risk of serious adverse effects, but probably increases the total number of adverse events in patients with COVID-19.

It is important to note that the clinical status of the patients differed between trials. Also, patients received various additional treatments, including other pharmacologic interventions such as interferon, corticosteroids, antibiotics, and immunoglobulin therapy.

In the last few weeks, multiple reviews assessing the role of this treatment for COVID-19 have been published[42],[43],[44]. However, due to the high speed at which new primary studies are being published, they are all missing relevant evidence. Furthermore, low certainty evidence leads to more considerable variability in recommendations, so it is not surprising that lopinavir/ritonavir has not been recommended in a recently published evidence-based guideline[45]. The authors indicated that there is not enough evidence of benefit for lopinavir/ritonavir in patients with COVID-19, while there is evidence of considerable harm.

Systematic reviews are the gold standard to collect and summarize the available evidence regarding a scientific question. However, the traditional model for conducting reviews has several limitations, including high demand for time and resources[46] and rapid obsolescence[47]. Amidst the COVID-19 crisis, researchers should make their best effort to answer the urgent needs of health decision-makers without giving up scientific accuracy. Information is being produced at a vertiginous speed[48], so alternative models are needed.

One potential solution to these shortcomings is to conduct rapid reviews, a form of knowledge synthesis that streamlines or omits specific methods of a traditional systematic review in order to move faster. Unfortunately, in many cases, this rapidity comes at the cost of quality[49]. Furthermore, they do not solve the issue of obsolescence. Living systematic reviews do address that issue[50]. They are continually updated by incorporating relevant new evidence as it becomes available, nonetheless, at a substantial effort. So, an approach combining these two models might prove more successful in providing the scientific community and other interested parties with evidence that is actionable, rapidly and efficiently produced, up to date, and of the highest quality[51].

This review is part of a larger project set up to put such an approach into practice. The project aims to produce multiple parallel living systematic reviews relevant to COVID-19 following the higher standards of quality in evidence synthesis production[15]. We believe that our methods are well suited to handle the abundance of evidence that is to come, including evidence on the role of lopinavir/ritonavir for COVID-19. We have identified multiple ongoing studies addressing this question, including 61 randomized trials, which will provide valuable evidence to inform researchers and decision-makers soon.

During the COVID-19 pandemic, we will maintain a living, web-based, openly available version of this review, and we will resubmit the review every time the conclusions change or whenever there are substantial updates. Our systematic review aims to provide high-quality, up-to-date synthesis of the evidence that is useful for clinicians and other decision-makers.

Appendix

Appendix 1
Appendix 2
Appendix 3
Appendix 4