A touch of history

A little over a year ago we began an ambitious joint project between Epistemonikos and Medwave, based on a new evidence summary format [1]. Collaborating in this venture was an easy decision for both parties, since we share a common interest in promoting evidence-based clinical decisions.

The Epistemonikos Summaries, or Living FRISBEEs (Living FRIendly Summary of the Body of Evidence using Epistemonikos), is a synthesis of all the systematic reviews on a single question, as well as the studies included by the reviews. In other words, it is a summary on the existing body of evidence regarding a specific clinical question, as it is understood from the viewpoint of evidence-based medicine [2]. Evidence-based medicine is a systematic approach toward clinical decision making that incorporates the best available evidence, along with clinical expertise and patient preferences [3].

The method used for the Epistemonikos Summaries allows balancing methodological rigor and much shorter output times than those needed for most existing evidence summaries and, more importantly, keeping them up to date. To achieve this, we use the Epistemonikos database, which is maintained through evidence searches on 30 databases and is continuously updated. With the help of sophisticated software and the cooperation among more than 600 people around the world, the database today boasts more than 70,000 systematic reviews, as well as 350,000 articles that “are worth knowing” (the meaning of the Greek word Epistemonikos). In short, this is the largest healthcare evidence database in the world, allowing us to rest assured in that the likelihood of overlooking important data is equal or lower than with any alternative method.  

The process begins with selection of the body of evidence to be summarized. This can be easily appreciated through the Epistemonikos evidence matrix [4], a visual interface that compares all systematic reviews addressing a similar question. Once the evidence matrix is complete, the summary author team extracts the relevant information from the systematic reviews, creating a new meta-analysis including all of the relevant studies as well as a Summary of Findings table using the GRADE method. Subsequently the summary is drafted, including a short description of the problem underlying the question, along with key messages expressed in plain language obtained from the Summary of Findings table using a structured method [5]. Finally, there is a section discussing the various considerations for decision-making. The summary conclusions are compared to those of other sources (the identified systematic reviews and the main clinical guidelines), and an estimation on the probability of the conclusions changing in the future should new evidence appear.

Albeit the methodology to carry out the summaries follows a rigorous process, the format is user-friendly, and the initial response from clinicians has been more than encouraging. Just as with other summary formats assessed in the literature, different users value different aspects, and require different levels of depth. For example, a significant number of clinicians will make a decision only using the key messages. Some will pay more attention to the findings, including the tables. Others will thoroughly review the summary component by component. Finally, a minority will also read the underlying evidence contained in the references. It is this “layers of depth” design that lies beneath the format of the summary. Initially, the key messages are highlighted, presenting the findings in no more than a few lines, including to what degree we can rely on the existing evidence.

For those needing more information, each section offers a quick overview of the issue underlying the question, including the main features of the existing studies, the summary of findings for the main outcomes (critical outcomes for decision-making, according to the terminology used by the GRADE method), and the clinical aspects to take into account for reaching a decision. Finally, for the few readers that need to analyze for themselves the original articles, the evidence matrix provides swift access for quick and easy comparison.

What we have accomplished so far
To date, 34 summaries have been completed, and the production rate is constantly rising, reaching close to two new summaries per week during this month. The summaries are now published as supplements, to make it easier for readers to review.

We already have groups working on practically all existing medical specialties, which little by little begin to become clear in the diversity of the topics addressed. These groups, supported methodologically by the Epistemonikos team, comprise more than a hundred novel researchers who have quickly grasped the method, and clinical experts. This last point should not be taken lightly, because while evidence-based medicine has been recognized since the beginning as necessary, it is not sufficient in and of itself for decision-making. Often, those who produce the evidence and those who use it are not in touch. This mix of components is perhaps a fundamental aspect in the initial success of the project: the active dialogue of clinical experience and evidence.

More important still from our point of view is that our summaries constitute the first example in the world of living evidence synthesis [6], which is to say it is continuously updated, as new data becomes available. One of the main problems nowadays is that systematic reviews become rapidly outdated, as do the clinical guidelines, not to mention the traditional narrative reviews or book chapters, or any other non-systematic way of summarising the evidence [7],[8].

At present, 20% of our summaries have needed to be updated within a year after publication. Some of them clearly illustrate the need for this approach. For instance, the first summary that needed updating was published in December 2014 [9]. Two days thereafter, a Cochrane systematic review appeared addressing the very same question. This new Cochrane review included two studies not identified by any of the previous reviews. Without incorporating this review to the summary, this would have been an extreme example of the fate that befalls any evidence synthesis sooner rather than later: obsolescence. With the proposed method, the new evidence was included in only 36 working minutes, and it was published the following day in Medwave.   

Another interesting example is that of a summary published in November 2014 singling out 14 systematic reviews related to the question [10]. Faced with such an amount of evidence, most readers would consider the chances of future evidence changing what we know are minimal. Nevertheless, a new review included 58 randomized controlled studies that had not been identified by any of the earlier reviews [11].

The counterpoint to this case is slated by the updating of a summary published in July 2015, for which we were able to detect a new systematic review merely two weeks after its publication [12]. The new review did not include any study that had not been identified in any of the six prior reviews, so the conclusions remained unchanged [13]. However, to reach this conclusion, it is necessary to review the recent studies and compare those to existing data. The differences among the various versions of the summary are clearly emphasized in each new publication, with the necessary details on the contribution made by the new evidence, and if it leads to changes vis-à-vis the earlier version. Thus the reader can make an informed decision as to whether any re-reading is required, and in turn, the clinician begins to gather knowledge in an orderly fashion, adding a brick to the already existing structure instead of rebuilding the entire construction.  

How to make the project a reality
In order to update in the shortest possible time, a coordinated effort is required from multiple players. Any weak link in the chain undermines the final objective.

The first link in the chain is the software designed by Epistemonikos, which makes it possible to automatically pick up new systematic reviews that are relevant to the query. These are quickly assessed by a team of authors, who include them as fast as possible and create a new version of the summary. Subsequently the Epistemonikos editorial team internally performs the peer review, and finally sends it to Medwave for publication.

In the editorial with which we began this series, we justified the decision –from the viewpoint of Epistemonikos– in selecting Medwave as partner in this ambitious project [1]. The attributes that led to the selection of Medwave from the beginning have not only held true after a year’s work, but are continuously improving: editorial agility, technical editing, bibliographical reference standardization, bilingual publication, free access for readers, a carefully designed and responsive website, and finally, efficient outreach including social networks, among others.

Moreover, the role of Medwave in this venture goes beyond the regular duties of a scientific journal publisher. On the one hand, a dexterous editorial process is required to reduce as much as possible the time between publication of new evidence and publication of a new summary. This task usually takes months or even years with traditional journals. On the other hand, there is significant difficulty in pointing out clearly to the user –both in the journal website and in the main databases where the journal is indexed– whether the article they are reading is indeed the most updated version or not. Medwave has provided solutions to these problems, allowing the reader to clearly identify which version they are dealing with (Figure 1).

Figure 1. Versions of Epistemonikos summaries in PubMed/MEDLINE

None of the above would be possible if it were not for the vision and mission alignment between Epistemonikos Foundation and Medwave, the latter with its corresponding and supporting Medwave Foundation: to provide information to everyone who may need it, so that the best health decisions can be made. Without shedding the rationale of scientific publication and of academia, it would be unthinkable to summarize today with such pride the results of this first year of joint work, and firmly state that this new way of accessing reliable and user-friendly evidence is here to stay.