Published on November 27, 2014 | http://doi.org/10.5867/medwave.2014.10.6044
A new way to find evidence quickly and efficiently
Una nueva forma para encontrar evidencia de manera rápida y eficiente
Abstract
Evidence coming from scientific research helps us make better decisions in relation with the care we provide to our patients. Some evidence is more reliable than other is, and systematic reviews are considered the best available evidence for answering questions that compare two or more alternatives of care, as long as they are well conducted and up-to-date. There has lately been a fast growth of the amount of available systematic reviews, so users need to choose between several possible options. Epistemonikos database is a highly efficient search resource, since it groups the larger number of systematic reviews worldwide, and allows accessing the body of evidence for a specific question in a simple and intuitive way. Additionally, it offers tools that enable the choice among different systematic reviews and for keeping the evidence for a question updated with minimal effort.
As discussed in the first article of this series (doi: 10.5867/medwave.2014.05.5966), there are instances where the evidence from scientific studies helps us make better decisions about the care we provide for our patients [1].
These instances are characterized by the fact that we are able to identify two or more care alternatives for a specific clinical setting, and which can be translated into a specific clinical question [2] that usually comprises the following elements: a Population (typically a disease, e.g. a population of patients with diabetes); a clinical course of Action to be assessed (a therapeutic Intervention or a diagnostic Test); a Comparison (the alternative to which the assessed action is compared, e.g. Is test A better than B?, or Is medication A superior to administering nothing at all?); and the Aim being pursued (in the case of therapy decisions this aim is the Outcome; e.g. to reduce mortality or improve quality of life).
What sort of evidence to look for
Systematic reviews were invented to deal with the difficulty in having several studies answering similar questions ‒often with discordant results‒ thus enabling what we call evidence based medicine [3]. A good systematic review constitutes the best evidence (a more reliable study design or with a higher probability of being true), at least at the time of publication, since it synthesizes all existing information for a specific question by going through a process that minimizes bias and error. Systematic reviews are essential in evidence based practice, and therefore should be the main aim of our search [4]. Unfortunately, with the increasing output of information these become outdated relatively quickly, and it is difficult for users to know if they are still valid unless compared to other similar reviews or by conducting an exhaustive review of the literature [5],[6].
How to select the “best evidence”
Contrary to what occurred years back, today we have scientific evidence (including systematic reviews) for the vast majority of decisions that clinicians make in their regular practice, and the volume of knowledge output is multiplying at great speed [7]. It could be said that we are in full “medical knowledge transition” from the lack of evidence to excessive evidence, especially in the topics of greater interest to clinicians and researchers. For this reason, today it is especially relevant to decide which information is the most reliable or, in practical terms, which systematic review is the most reliable among all those available.
If we are able to obtain all relevant systematic reviews, we will have the best possible approximation (without performing a new systematic review) to the “body of evidence” related to our question [4].
The three main elements proposed to determine which the best systematic review is, are:
- Date of publication (or, ideally, the date of the search conducted by the systematic review): for obvious reasons, more recent reviews have a greater probability of including the updated body of evidence for a question.
- Number of studies included: the greater the amount of studies (germane to the question asked), the greater the probability of it being representative of the body of evidence.
- Quality of the review: better quality reviews have a greater probability of finding the body of evidence and of avoiding errors and biases that lead to conclusions which are far from the truth.
Where to search for evidence
There are multiple search resources, such as general search engines (Google, Wikipedia), health research databases (PubMed), and databases specialized in health evidence. This is a rapidly changing area, where there is still no definitive answer to “which is the best place to search for?” In this article we propose to always begin with database specialized in health evidence Epistemonikos (8). For a more comprehensive review of the various alternatives and approximations to evidence, we recommend reading the respective chapter in “Users' Guides to the Medical Literature” [9].
Epistemonikos (“what is worth knowing”, in Greek) is a multilingual, collaborative database, which attempts to group together all relevant evidence for health decision-making, and it is currently the greatest collection of systematic reviews and other types of evidence worldwide. A unique characteristic of this database is that it connects different articles answering a similar question, and it offers visual tools to help determine which is possibly the best available evidence [10].
How to choose the terms for a search using Epistemonikos
A good search begins with the correct formulation of the question [2]. To transform the clinical question into a search strategy it is sufficient to follow these recommendations:
- Not all components of the question should be used in the search. A search with few terms is generally safer. Typically, for a treatment question, one term is sufficient for the intervention and another for the population of interest.
- Comparison is only used if it is an active comparison. It is unnecessary ‒and could even be counterproductive‒ to use terms such as ‘placebo’, ‘no treatment’, ‘standard therapy’, and so on.
- The aim (or Outcome) is not usually part of the search strategy. There are multiple relevant outcomes for each question, and how they are denominated and measured varies greatly.
- Avoid terms that are not specific or unrelated to the question components. For example, terms such as ‘treatment’, ‘prevention’, or ‘diagnosis’.
- Rarely use Boolean operators: Epistemonikos, same as most sites, allows using the main logical or Boolean operators to combine terms cases (see Figure 1). They can be useful in selected. The advanced search interface allows traditional Boolean strategies.
Full size
Systematic reviews in context
Not all systematic reviews are the same. Hence we should learn to recognize how reliable they are [11]. Looking to see in which journal they are published or who is the author is insufficient. The only exception are the systematic reviews conducted by members of the Cochrane Collaboration, which ‒on average‒ are of better quality than non-Cochrane reviews [12] (we will further elaborate on this in a future article).
Furthermore, systematic reviews are not designed for users unaccustomed to technical research language, and do not take into account all the elements necessary for decision-making. This is why it is often of much use to read what is known as a “structured abstract” of the review (abstract and independent critical analysis by another group).
On the other hand, various therapeutic alternatives or courses of action are ever more frequently found for one same clinical situation, and hence the answer to “which is the best management option for condition X?” will often require assessing multiple systematic reviews. Overviews of systematic reviews and clinical practice guidelines are a good alternative in these cases [13]. Nevertheless, given the breadth of their approach, producing them is expensive and they are outdated more quickly than individual reviews.
How to access the best evidence in 4 steps using Epistemonikos
Clinical scenario. Imagine you are an obstetrician in a public hospital and are concerned about the lack of beds in your unit. A significant number of beds are occupied by women recovering from caesarian section or other surgery. A colleague mentions that chewing gum can speed-up recovery of postoperative ileus in abdominal surgery, and hence patients begin eating earlier and could be discharged more quickly. Perhaps it is also effective in caesarian sections.You express certain skepticism at the proposal and decide to look for the evidence. You first run a search on Google, which provides a few results but nothing you consider reliable. You subsequently attempt a search on PubMed and find some things, but the few articles that seem convincing require payment before reading. Somewhat discouraged, you tell your colleague that you do not believe there is sufficient evidence on his proposal. He proposes doing a search on Epistemonikos (www.epistemonikos.org), a new health evidence search engine. |
Step 1: run a simple search
One characteristic of Epistemonikos is that it can perform a search in several languages, obtaining results that include not only articles in that language but all the articles. A simple search such as “chewing-gum abdominal surgery” or “abdominal postoperative chewing-gum” is sufficient to find relevant information. If the terms are known in English they can also be used, or a combination of terms in English and any other of the 9 languages supported by the database.
Step 2. identify a systematic review that answers your question
From the search results it is necessary to identify a systematic review that answers the question. For this purpose, the most efficient way is to use the connections among articles in the database. For example, if a study is found that answers the question, we can follow the connection toward the systematic reviews that include the study (see Figure 2).
The filter for systematic reviews to the left of the search engine can also be used.
One very important point is that systematic reviews have been selected by collaborators following strict criteria, so the user does not need to determine if it is really a systematic review or not. Traditional search engines such as PubMed provide “possible systematic reviews”, and hence the user should discriminate if the result is or not a systematic review.
Full size Step 3: analyze the body of evidence for the question
Once a review has been selected we should use the tools offered by Epistemonikos to assess other systematic reviews for the same question.
When accessing the article abstract (by clicking on the title) a diagram appears to the right showing the systematic reviews that answer a similar question and a link to the “matrix of evidence” tool (see Figure 3).
Full size The matrix of evidence shows all systematic reviews sharing primary studies with the original review, and all of the studies identified among these reviews (see Figure 4). The percentage of primary studies in common is an efficient indicator of the degree of similarity of the question. In other words, if two reviews share a study, they probably answer a related question; if they share many studies they probably answer the same question.
Full size The user can edit the matrix of evidence to reflect the question asked. Typically, reviews and studies answering related questions but not exactly the same question should be removed.
Matrixes can be saved to the user account, and whenever new related evidence appears, an automatic warning is shown, making it very easy to keep the “body of evidence” updated for a specific question, situating the new evidence in this context (see Figure 5).
Full size Step 4. situate the best evidence in context
Not all systematic reviews are equally reliable (even if including all existing studies for the question asked) and are not specifically designed for a user unaccustomed to technical research language. Furthermore, these reviews do not consider all the elements necessary for decision-making. That is why it is often more useful for us to read a structured abstract of the review (abstract and independent critical analysis by another group), an overview of systematic reviews, or a clinical practice guideline. All these types of evidence are interconnected in Epistemonikos.
Solution to the clinical scenario. You run a search in Epistemonikos using “chewing-gum abdominal surgery”. The first systematic review that appears is clearly relevant (Figure 2). On this basis you create a matrix ofevidence showing 14 systematic reviews. One of these is irrelevant and then you remove it from the matrix, ending up with 13 reviews that include 26 studies in total (Figure 4). Considering there is so much evidence, you decide to focus on the articles that directly assess the effect of chewing-gum on caesarian section. You remove the reviews and studies assessing other surgery types and create a matrix that encompasses the body of evidence for this new question. You then use the filter for study design, and check there are five systematic reviews that in total include seven randomized control studies that precisely answer your question (Figure 5). Only one review, published a few months before, includes all existing studies. You realize this review is not yet included in guidelines or overviews of systematic reviews, and there is not a structured abstract. You are convinced this is the best evidence for your question and proudly return to your colleague. You are now ready for the next step in the evidence informed decision-making process. |
Notes
Interests
Epistemonikos foundation is an organization that intends to bring information closer to health care decision-makers through the use of technology. Its main output is the Epistemonikos database (www.epistemonikos.org).
The authors have completed the ICMJE. IN declares not to have potential conflicts of interest with the subject of this article and has received fees for methodological support in the preparation of clinical practice guidelines in the last 36 months by the American Gastroenterological Association and the Kingdom of Saudi Arabia. JC declares not having potential conflicts of interest with the subject of this article.
