Login | Register | Forgot your password? | Contact | Español |
Email: Password:

Medwave se preocupa por su privacidad y la seguridad de sus datos personales.
Para poder enviarle a su dirección de correo electrónico su contraseña, es necesario que ingrese su e-mail.


Review article
Medwave 2018;18(7):e7354 doi: 10.5867/medwave.2018.07.7354
Syntheses of biomedical information: narrative reviews, systematic reviews and emerging formats
Juan Víctor Ariel Franco, Marcelo Arancibia, Daniel Simancas-Racines, Eva Madrid
References | Download PDF |
To Download PDF must login.
Print | A(+) A(-) | Easy read

Key Words: review, evidence-based medicine

Abstract

Biomedical information dissemination has expanded exponentially, and this can represent a challenge for those health professionals who wish to obtain high quality and relevant integrated information. Reviews, in their different formats, are tools that can address this problem. This article describes the main types of syntheses of biomedical information, their structures, their usefulness, and presents the latest information synthesis formats that were developed by different organizations committed to this purpose.


 
Introduction

The number of articles a health professional should read in order to keep up to date has grown exponentially, as David Sackett mentioned in his book[1],[2]. Archie Cochrane had already foreseen that doctors would need "periodical summaries by specialty" to assist the healthcare professional in making decisions[3]. Taking into account the cognitive complexity involved in the task of adequate reading of scientific articles, in terms of the development of a critical and constructive capacity, and also the knowledge about the methods in biomedical research and its potential sources of important biases, including the manipulation and concealment of data, this makes “the critical assessment of scientific evidence” an arduous task for the healthcare professionals when it comes to answering their clinical questions and stay up-to-date. Literature reviews, in their different formats, try to bring the knowledge of biomedical evidence somewhat closer.

Reviews are articles that synthesize scientific information using primary publications of original research as a source. Following the scheme of evidence-based medicine[4],[5], some key steps for a review would include:

1) Formulating a clinical structured question (“PICO”: patient, intervention, comparison and outcome)/topic of the review
2) Searching bibliographic scientific information
3) Critical assessment of the findings
4) Formulating conclusions based on the synthesized evidence

Considering this, there is a wide range of articles denominated "reviews," but with substantial differences in their methods of elaboration and proposed objectives[6]. Thus, the objectives of this analysis are: to outline some differences among the different types of reviews and to describe the basic aspects of each one of them. We will also point out the role of some organizations involved in the development and methodology of synthesis of biomedical information.

Types of reviews

Based on the thoroughness and the reproducibility in which the aforementioned steps are conducted, we can subclassify reviews into “narrative” and “systematic.”

1. Narrative reviews
These are reviews that synthesize a subject without a specific or previously declared methodology. Unlike systematic reviews, narrative reviews select the evidence to synthesize in a way that is not reproducible and without an exhaustive searching. Therefore, the reader or researcher cannot follow the steps taken by the author to reach the same conclusions. In addition, by not using a well-defined methodology there is a high risk that the selected information is biased or manipulated, limiting the conclusions and subtracting confidence in the results of this type of study.

However, narrative reviews occupy an important place in continuing education, as they update readers on specific topics. They may have different denominations according to specific goals[6](Table 1), and they differ from systematic reviews in multiple structural and procedural aspects[7] (Table 2).

Narrative reviews can be useful when answering broad questions ("global deficit" questions)[8] related to basic concepts, e.g. "What is Ehlers-Danlos syndrome?", "what is the natural course of the chickenpox infection?" They can also be a way of addressing the opinion of individuals or groups referring to a subject. In this sense, a typical example of narrative review is a chapter of a medical book. However, when answering more specific questions using narrative reviews ("gap" type questions, e.g. "Does the use of antivirals in adults with chickenpox reduce the incidence of secondary complications?")[8], the synthesized evidence may be incomplete, selected in a biased way (“cherry picking”)[9] and difficult to interpret because of the lack of assessment of their quality and applicability.

Narrative reviews are often developed by the disciplinary experts on the subject, but they do not usually include methodologists in the development of this type of work. A single narrative review may incorporate different populations of study: human studies, animal models or in vitro studies. They can address topics about etiology, diagnosis, treatment and/or prognosis.

With the advent of systematic reviews (see below), this type of reviews has sometimes been placed in a higher hierarchy than narrative reviews, especially in biomedical journals. Nevertheless, narrative reviews provide a deep, reflective and thoughtful understanding of the issues with which the health care professional struggles daily[10]. This is particularly important when reflecting the elements that constitute the complexity of the clinical question. For example, a therapeutic question can be "bounded" in its spectrum but requires synthesizing information from dozens of clinical trials (for which a systematic review has methods to ensure the transparency and reproducibility of its findings). However, a question about the implications of a given therapeutic can be "bounded" in the evidence underpinning its response, but it may require a great thoughtful and critical elaboration based on philosophical, social, and nosological sources, among others.

Table 1. Different denominations for narrative reviews

Table 2. Differences between a narrative review and a systematic review.

2. Systematic reviews
These reviews help when answering to a specific clinical question or synthesize a topic using a reproducible methodology, consigned a strict compliance with a research protocol established a priori and, in many cases, published in specialized databases, this protocol includes eligibility criteria for the studies to include, a section of methods and statistical analysis.

This type of review assimilates the four steps of evidence-based medicine and links specific tools for its proper and rigorous development[4]. In Table 3 there is a summary of the relationship between evidence-based medicine and the methodology of systematic reviews.

Table 3. Steps of evidence-based medicine and systematic reviews meth-odology.

A central aspect in the systematic reviews of literature, is that they are based on the findings of systematic searches of all possible sources (exhaustive search of published and unpublished evidence), which attempt to minimize selection bias, unlike narrative reviews, in which the intuition of the author, his/her experience and even expert opinions might influence the findings through the phenomenon of “cherry picking”[11]. This does not imply that the vision of an expert on the subject can be of vital importance in the methods and conduct of systematic reviews, although there is some debate on the role of these experts and the possibility of introducing biases in the review process[12].

Systematic reviews are very useful for the aforementioned "gap" type clinical questions. These questions can be associated, for example, to the assessment of the effectiveness and safety of therapeutic/preventive options, the evaluation of the diagnostic accuracy or the identification of the prognostic factors of a disease (In Table 4 some examples are shown). In this way, systematic reviews emerged from the framework of the evidence-based medicine, integrating all the necessary steps for the incorporation of scientific information in the decision-making processes in healthcare.

Table 4. Clinical questions and PICO-format formulation for systematic reviews.

Systematic reviews use the primary data of scientific research relevant to the clinical question as a source of information, which may be published or unpublished (grey literature). Therefore, it prioritizes information from clinical trials for therapeutic questions, cohort studies for prognostic questions and cross-sectional studies for those related to diagnostic accuracy.

2.1 Quality of systematic reviews
We refer to quality in systematic reviews when answering the question “how adequately was carried out this review?" The most frequently used instrument to address this question is AMSTAR (A Measurement Tool To Assess Systematic Reviews)[13], which has recently been updated to its version AMSTAR 2[14] (Table 5) . There is also another tool called ROBIS (Risk Of Bias In Systematic Reviews)[15],[16], which attempts to determine whether systematic reviews may have incurred biases during their conduct, the so-called "meta-biases"[17] that could affect their conclusions.

Table 5. AMSTAR 2 criteria for the assessment of the quality of systematic reviews.

2.2 Systematic reviews and Cochrane
In 1993 a charity organization was founded, originally called "The Cochrane Collaboration," now called "Cochrane." This organization is comprised of more than 43 000 volunteer health professionals around the world, and its mission is “to promote evidence-informed health decision-making by producing high-quality, relevant, accessible systematic reviews and other synthesized research evidence”[18]. Systematic review conducted by this collaboration usually have better quality than systematic reviews conducted outside its influence[19]. Cochrane has two methodological guides that attend the review process: the Cochrane Handbook4 and the Methodological Expectations (MECIR)[20]. Even so, non-Cochrane reviews using this rigorous methodology for information synthesis can also have high quality. Cochrane reviews, despite having a rigorous process of evaluation during processing, may also suffer from methodological limitations[19],[21],[22],[23],[24],[25].

Cochrane has a geographic organization in centers of diverse hierarchy located in different parts of the world which aim to contribute to the mission and the goals of Cochrane. In the spanish-speaking countries, the Cochrane Iberoamerican Network stands out, which is a very active branch in the production and dissemination of systematic reviews. The network has collaborative work projects among countries, such as the hand-search project[26], that attempts to identify unpublished or difficult-to-access clinical trials in order to reduce dissemination bias.

One of the most important methodological innovations Cochrane has incorporated in its reviews is the GRADE methodology[27]. GRADE is a system of critical evaluation of the evidence that allows to synthesize the confidence in the findings with a structured and transparent methodology, and by means of a simplified language, which allows the integration of the quality of the evidence with the results, summarizing it in "high," "moderate," "low," and "very low," quality. This gradation of evidence has implications both in the clinical field (which relates to the level of uncertainty at the time of making a clinical decision) and research (when analyzing the area in which this uncertainty requires enlightening studies).

2.3 Other organizations that develop systematic reviews
There are other organizations like Cochrane that promote the production of high-quality systematic reviews on various topics. Some examples include the BEME collaboration (www.bemecollaboration.org), focused on medical education, and the Campbell Collaboration (www.campbellcollaboration.org), focused on social topics. Systematic reviews have also gained relevance in other fields such as basic and translational research[28]. Two organizations that have promoted initiatives for the development of systematic reviews in these areas are CAMARADES (Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies – www.dcn.ed.ac.uk/camarades/) and SYRCLE (Systematic Review Center For Laboratory Animal Experimentationwww.syrcle.nl).

3. Other emerging forms of information synthesis
There are other review formats that use systematic (reproducible) methods to formulate documents that assist in health decision-making:

3.1. Umbrella Reviews: They are reviews that compile findings from a group of systematic reviews. Usually they do not follow a PICO format, but address to a more general topic, e.g. interventions for enuresis[29]. This “umbrella review" compiles data from several systematic reviews covering different strategies for the management of a condition (Figure 1).

Figure 1. Example of overview/umbrella review.

3.2 Scoping reviews ("Scope reviews"): They are reviews that compile evidence on a subject, generally with a broad "scope," without critical assessment of the evidence30. However, there is great variability in the methods used to perform them and these could be flexible[31].

3.3 Global evidence mapping ("evidence mapping"): It is a tool that allows the identification of the studies or reviews within a certain topic, and does not make a synthesis of the findings, but locates the evidence in individual questions of a theme[32]. It is particularly useful for identifying "evidence gaps " in which there are no studies evaluating a particular intervention[33] (Figure 2).

Figure 2. Evidence map fragment: adolescent sexual and reproductive health.

3.4 Rapid reviews: They are similar to systematic reviews in their structure, however, their methods are simplified to achieve the formulation of a report within a limited period of time (usually 10 to 12 weeks), to rapidly assist decisions[34].

3.5. Realist reviews: Incorporate qualitative research concepts, which comprises the understanding the complexity of certain phenomena in health (education, communication and other complex interventions). They focus on the contextual component of the interventions and the mechanisms that lead to a particular outcome (context + mechanism = outcome). To build this explanatory approach, these reviews use a theoretical framework of reference for the qualitative analysis[35].

Conclusions

There are different types of health reviews. Narrative reviews, which do not use systematic methods, often have great value when synthesizing information on broad subjects or background. They can provide an interpretative and reflective view of a specific problem, nourished by various bibliographical sources. On the other hand, systematic reviews answer a question focused on a reproducible methodology that tries to minimize and provide transparency to biases in the body of evidence. These and other new forms of review, together with the information of the context and the patient, attend the process of decision-making in health.

Notes

From the editor
The authors originally submitted this article in Spanish and subsequently translated it into English. The Journal has not copyedited this version.

Role of the authors
JF, EM, DS: Conceptualization. JF and MA: Writing of the original manuscript. EM and DS: Manuscript review and editing. MA: Project management. EM and MA: Publication financing.

Conflicts of Interest
The authors declare that they have no conflicts of interest.

Funding source
This project did not have specific funding sources. The publication was financed by the Universidad de Valparaíso. The institutions contributed to the working time of each one of the authors.

Ethical approval
This project does not require review by ethics Committee.

Table 1. Different denominations for narrative reviews
Table 2. Differences between a narrative review and a systematic review.
Table 3. Steps of evidence-based medicine and systematic reviews meth-odology.
Table 4. Clinical questions and PICO-format formulation for systematic reviews.
Table 5. AMSTAR 2 criteria for the assessment of the quality of systematic reviews.
Figure 1. Example of overview/umbrella review.
Figure 2. Evidence map fragment: adolescent sexual and reproductive health.
Licencia Creative Commons Esta obra de Medwave está bajo una licencia Creative Commons Atribución-NoComercial 3.0 Unported. Esta licencia permite el uso, distribución y reproducción del artículo en cualquier medio, siempre y cuando se otorgue el crédito correspondiente al autor del artículo y al medio en que se publica, en este caso, Medwave.

 

La divulgación de información biomédica se expande de manera exponencial, lo que representa un desafío para aquellos profesionales de la salud que deseen obtener información integrada de alta calidad y relevancia. Las revisiones, en sus distintos formatos, son herramientas que pueden abordar este problema. Este artículo describe los principales tipos de síntesis de información biomédica, sus estructuras, la utilidad de cada una de ellas, y presenta los últimos formatos desarrollados por las diferentes organizaciones que se dedican a la síntesis de información.

Authors: Juan Víctor Ariel Franco[1], Marcelo Arancibia[2], Daniel Simancas-Racines[3], Eva Madrid[2]

Affiliation:
[1] Instituto Universitario Hospital Italiano, Centro Cochrane Asociado, Argentina
[2] Centro Interdisciplinario de Estudios en Salud (CIESAL), Escuela de Medicina, Universidad de Valparaíso, Centro Cochrane Universidad de Valparaíso, Valparaíso, Chile
[3] Centro de Investigación de Salud Pública y Epidemiología Clínica (CISPEC), Centro Asociado Cochrane de Ecuador, Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador

E-mail: marcelo.arancibiame@uv.cl

Author address:
[1] Facultad de Medicina
Universidad de Valparaíso
Angamos 655
Viña del Mar
Chile

Citation: Franco JVA, Arancibia M, Simancas-Racines D, Madrid E. Syntheses of biomedical information: narrative reviews, systematic reviews and emerging formats. Medwave 2018;18(7):e7354 doi: 10.5867/medwave.2018.07.7354

Submission date: 31/8/2018

Acceptance date: 12/11/2018

Publication date: 27/11/2018

Origin: not requested

Type of review: reviewed by two external peer reviewers, double-blind

PubMed record


 

Errata

1. Citation and affiliations are corrected in this article | Link |

Comments (0)

We are pleased to have your comment on one of our articles. Your comment will be published as soon as it is posted. However, Medwave reserves the right to remove it later if the editors consider your comment to be: offensive in some sense, irrelevant, trivial, contains grammatical mistakes, contains political harangues, appears to be advertising, contains data from a particular person or suggests the need for changes in practice in terms of diagnostic, preventive or therapeutic interventions, if that evidence has not previously been published in a peer-reviewed journal.

No comments on this article.


To comment please log in

Medwave provides HTML and PDF download counts as well as other harvested interaction metrics.

There may be a 48-hour delay for most recent metrics to be posted.

  1. Sackett DL. Evidence-based medicine : how to practice and teach EBM. Churchill Livingstone; 1997.
  2. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010 Sep 21;7(9):e1000326. | CrossRef | PubMed |
  3. Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to "Effectiveness and Efficiency: Random Reflections on the Health Services" 1972. Control Clin Trials. 1989 Dec;10(4):428-33. | PubMed |
  4. Green S, Higgins JP, Alderson P, et al. Chapter 1: Introduction. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration 2011. handbook.cochrane.org [on line] . | Link |
  5. Students4BestEvidence. What are the key steps in Evidence-Based Medicine? [on line] | Link |
  6. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. | CrossRef | PubMed |
  7. Pae CU. Why Systematic Review rather than Narrative Review? Psychiatry Investig. 2015 Jul;12(3):417-9. | CrossRef | PubMed |
  8. Kopitowski K. Medicina Basada en la Evidencia ¿Cómo formular una pregunta efectiva ? Evid en Atención Primaria. 2002;5:92-4. | Link |
  9. Kopitowski K. La falacia de la evidencia incompleta o “cherry picking”. Evid Act Pr Ambul. 2016;19:35-6. | Link |
  10. Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018 Jun;48(6):e12931. | CrossRef | PubMed |
  11. Huw T, Crombie D. What is a systematic review? Evidence-based medicine. [on line] | Link |
  12. Gøtzsche PC, Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ 2012;345:e7031. | CrossRef | PubMed |
  13. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007 Feb 15;7:10. | PubMed |
  14. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017 Sep 21;358:j4008. | CrossRef | PubMed |
  15. Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. [ROBIS: a new tool to assess risk of bias in systematic reviews was developed]. Recenti Prog Med. 2018 Sep;109(9):421-431. | CrossRef | PubMed |
  16. University of Bristol. ROBIS : Tool to assess risk of bias in systematic reviews. 2015. .bristol.ac.uk.[on line]. | Link |
  17. Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med. 2011 Jul 5;155(1):61-2. | CrossRef | PubMed |
  18. Cochrane. Our vision, mission, and principles. cochrane.org [on line]. | Link |
  19. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016 May 24;13(5):e1002028. | CrossRef | PubMed |
  20. Higgins JPT, Lasserson T, Chandler J, et al. Methodological Expectations of Cochrane Intervention Reviews (MECIR). 2016:1-65. [on line] | Link |
  21. Franco JVA, Garrote VL, Escobar Liquitay CM, Vietto V. Identification of problems in search strategies in Cochrane Reviews. Res Synth Methods. 2018 Sep;9(3):408-416. | CrossRef | PubMed |
  22. Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. J Med Libr Assoc. 2009 Jan;97(1):21-9. | CrossRef | PubMed |
  23. Castellini G, Bruschettini M, Gianola S, Gluud C, Moja L. Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis. Syst Rev. 2018 Jul 28;7(1):110. | CrossRef | PubMed |
  24. Propadalo I, Tranfic M, Vuka I, Barcot O, Pericic TP, Puljak L. In Cochrane reviews risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance. J Clin Epidemiol. 2018 Oct 9. pii: S0895-4356(18)30625-5. | CrossRef | PubMed |
  25. Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006 Oct;59(10):1057-63. | PubMed |
  26. Cochrane. Proyecto de búsqueda manual | Cochrane Iberoamérica. cochrane.org [on line]. | Link |
  27. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, et al. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8. | CrossRef | PubMed |
  28. Ritskes-Hoitinga M, Leenaars M, Avey M, Rovers M, Scholten R. Systematic reviews of preclinical animal studies can make significant contributions to health care and more transparent translational medicine. Cochrane Database Syst Rev. 2014 Mar 28;(3):ED000078. | CrossRef | PubMed |
  29. Russell K, Kiddoo D. The Cochrane Library and nocturnal enuresis; an umbrella review. Evidence-Based Child Heal A Cochrane Rev J. 2006;1(1):5-8. | CrossRef |
  30. Peterson J, Pearce PF, Ferguson LA, Langford CA. Understanding scoping reviews: Definition, purpose, and process. J Am Assoc Nurse Pract. 2017 Jan;29(1):12-16. | CrossRef | PubMed |
  31. Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014 Dec;5(4):371-85. | CrossRef | PubMed |
  32. Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016 Feb 10;5:28. | CrossRef | PubMed |
  33. International Initiative for Impact Evaluation. Adolescent Sexual and Reproductive Health Evidence Gap Map. gapmaps.3ieimpact.org [on line] | Link |
  34. Garritty C, Stevens A, Gartlehner G, King V, Kamel C; Cochrane Rapid Reviews Methods Group. Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses. Syst Rev. 2016 Oct 28;5(1):184. | PubMed |
  35. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005 Jul;10 Suppl 1:21-34. | PubMed |
Sackett DL. Evidence-based medicine : how to practice and teach EBM. Churchill Livingstone; 1997.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010 Sep 21;7(9):e1000326. | CrossRef | PubMed |

Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to "Effectiveness and Efficiency: Random Reflections on the Health Services" 1972. Control Clin Trials. 1989 Dec;10(4):428-33. | PubMed |

Green S, Higgins JP, Alderson P, et al. Chapter 1: Introduction. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration 2011. handbook.cochrane.org [on line] . | Link |

Students4BestEvidence. What are the key steps in Evidence-Based Medicine? [on line] | Link |

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. | CrossRef | PubMed |

Pae CU. Why Systematic Review rather than Narrative Review? Psychiatry Investig. 2015 Jul;12(3):417-9. | CrossRef | PubMed |

Kopitowski K. Medicina Basada en la Evidencia ¿Cómo formular una pregunta efectiva ? Evid en Atención Primaria. 2002;5:92-4. | Link |

Kopitowski K. La falacia de la evidencia incompleta o “cherry picking”. Evid Act Pr Ambul. 2016;19:35-6. | Link |

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018 Jun;48(6):e12931. | CrossRef | PubMed |

Huw T, Crombie D. What is a systematic review? Evidence-based medicine. [on line] | Link |

Gøtzsche PC, Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ 2012;345:e7031. | CrossRef | PubMed |

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007 Feb 15;7:10. | PubMed |

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017 Sep 21;358:j4008. | CrossRef | PubMed |

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. [ROBIS: a new tool to assess risk of bias in systematic reviews was developed]. Recenti Prog Med. 2018 Sep;109(9):421-431. | CrossRef | PubMed |

University of Bristol. ROBIS : Tool to assess risk of bias in systematic reviews. 2015. .bristol.ac.uk.[on line]. | Link |

Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med. 2011 Jul 5;155(1):61-2. | CrossRef | PubMed |

Cochrane. Our vision, mission, and principles. cochrane.org [on line]. | Link |

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016 May 24;13(5):e1002028. | CrossRef | PubMed |

Higgins JPT, Lasserson T, Chandler J, et al. Methodological Expectations of Cochrane Intervention Reviews (MECIR). 2016:1-65. [on line] | Link |

Franco JVA, Garrote VL, Escobar Liquitay CM, Vietto V. Identification of problems in search strategies in Cochrane Reviews. Res Synth Methods. 2018 Sep;9(3):408-416. | CrossRef | PubMed |

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. J Med Libr Assoc. 2009 Jan;97(1):21-9. | CrossRef | PubMed |

Castellini G, Bruschettini M, Gianola S, Gluud C, Moja L. Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis. Syst Rev. 2018 Jul 28;7(1):110. | CrossRef | PubMed |

Propadalo I, Tranfic M, Vuka I, Barcot O, Pericic TP, Puljak L. In Cochrane reviews risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance. J Clin Epidemiol. 2018 Oct 9. pii: S0895-4356(18)30625-5. | CrossRef | PubMed |

Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006 Oct;59(10):1057-63. | PubMed |

Cochrane. Proyecto de búsqueda manual | Cochrane Iberoamérica. cochrane.org [on line]. | Link |

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ, et al. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8. | CrossRef | PubMed |

Ritskes-Hoitinga M, Leenaars M, Avey M, Rovers M, Scholten R. Systematic reviews of preclinical animal studies can make significant contributions to health care and more transparent translational medicine. Cochrane Database Syst Rev. 2014 Mar 28;(3):ED000078. | CrossRef | PubMed |

Russell K, Kiddoo D. The Cochrane Library and nocturnal enuresis; an umbrella review. Evidence-Based Child Heal A Cochrane Rev J. 2006;1(1):5-8. | CrossRef |

Peterson J, Pearce PF, Ferguson LA, Langford CA. Understanding scoping reviews: Definition, purpose, and process. J Am Assoc Nurse Pract. 2017 Jan;29(1):12-16. | CrossRef | PubMed |

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014 Dec;5(4):371-85. | CrossRef | PubMed |

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016 Feb 10;5:28. | CrossRef | PubMed |

International Initiative for Impact Evaluation. Adolescent Sexual and Reproductive Health Evidence Gap Map. gapmaps.3ieimpact.org [on line] | Link |

Garritty C, Stevens A, Gartlehner G, King V, Kamel C; Cochrane Rapid Reviews Methods Group. Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses. Syst Rev. 2016 Oct 28;5(1):184. | PubMed |

Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005 Jul;10 Suppl 1:21-34. | PubMed |