Abrir sesión | Regístrese | ¿Olvidó su contraseña? | Contacto | English |
Email: Clave:

Medwave se preocupa por su privacidad y la seguridad de sus datos personales.
Para poder enviarle a su dirección de correo electrónico su contraseña, es necesario que ingrese su e-mail.


Análisis
Medwave 2019;19(6):e7664 doi: 10.5867/medwave.2019.06.7664
La revisión sistemática como herramienta para la validación científica de propiedades saludables de los alimentos y factores alimentarios
Using systematic reviews in the scientific substantiation of health properties of foods and food constituents
Mariane Lutz, Marcelo Arancibia, Jana Stojanova
Referencias | Descargar PDF |
Para Descargar PDF debe Abrir sesión.
Imprimir | A(+) A(-) | Lectura fácil

Palabras clave: systematic review, randomized controlled trial, food, health

Abstract

Systematic literature reviews are one of the main methodologies used to substantiate the health properties of foods and food constituents purported to affect human physiology. This tool is based on scientific evidence obtained from correctly performed randomized controlled trials. Systematic reviews make it possible to conclude whether there is a causal relationship between food consumed and health effects observed, thus supporting the use of the term “functional foods.” We present and analyze the prinicpal characteristics of, and ways by which, systematic reviews can contribute to the regulatory approval of health claims directed to consumers.


 

Main messages

  • Systematic reviews of randomized clinical trials are the best tool available to evaluate the effect of a health intervention.
  • This methodological design can be applied to the evaluation of the physiological effects of the consumption of foods and food constituents.
  • Systematic reviews allow scientific substantiation of health claims associated with functional foods.

·         Systematic reviews of randomized clinical trials are the best tool available to evaluate the effect of a health intervention

·         This methodological design can be applied to the evaluation of the physiological effects of the consumption of foods and food constituents.

·         Systematic reviews allow scientific substantiation of health claims associated with functional foods.

Introduction

Foods are not only vehicles of nutrients and energy; they also provide pleasure, wellness, and, importantly, components that exert physiological actions beyond a nutritional contribution. These properties, designated healthy or functional, are due to "bioactive" chemical compounds contained in a food matrix and constitute a topic that has occupied the attention of researchers, consumers, the food industry and legislators for many years. In this context, the promotion of health claims through various marketing strategies is especially relevant. Foods referred to as functional and purported to possess health attributes should be scientifically substantiated. However, there is often a lack of robust evidence to support such claims. This situation has led regulatory organisms of different countries to establish clear criteria regarding this subject[1],[2].

To make a health claim associating the consumption of a food or food constituent with a beneficial effect related to a disease or health condition, the effect should first be demonstrated. This is the only means for claims to be accepted by regulatory agencies, such as EFSA (European Food Safety Agency) (http://www.efsa.europa.eu/), FDA (US Food and Drug Agency) (https://www.fda.gov/) or the Ministry of Health, Labor and Well-being of Japan (https://www.mhlw.go.jp/english/). In Chile, the agency responsible for approving health claims is the Ministry of Health (https://www.minsal.cl/), according to the Foods Sanitary Code (Decree 977/96 and amendments). For its part, the Codex Alimentarius (http://www.fao.org/fao-who-codexalimentarius/es/), the principal food regulation organism on a global level, defines a health claim as "any representation that states, suggests or implies that a relationship exists between a food, or a food constituent, and health." It adds: "Health and functional claims must be supported by sufficient evidence to justify them, not mislead but provide truthful information to help the consumer make healthy decisions and be supported by education directed to the consumer." Today, however, there is much promotion of food and food constituents attributing beneficial health properties, without proper scientific substantiation of these effects.

Systematic reviews consist of the search for, and compilation of, empirical evidence with pre-established criteria to answer a specific question of interest[3], and are a tool that supports evidence-based nutrition[4]. This article aims to describe systematic review methodology and its role in the validation of purported health effects of food products, whether they have been extracted and isolated from their source or as part of the original food matrix consumed habitually.

Contributions of different methodological designs to functional food research

Clinical epidemiology has provided a systematization of different methodological designs that address research questions. Observational studies, such as cross-sectional, case-control and cohort studies, provide relevant information; however, they do not allow determining causality and are susceptible to a variety of biases that may affect the results[5],[6]. On the physiological effects of foods and their evaluation, observational designs do not constitute the optimal methodology to determine causality.

If what is sought is to analyze a dietary intervention on a specific physiological condition, the design of choice is the randomized clinical trial, the term for experimental studies in clinical epidemiology[7],[8]. In randomized clinical trials, the exposure (food or food factor) is randomly assigned to a group of subjects, while a comparison group receives a similar product that does not contain the food factor (a placebo or a comparator). Randomization is a central element in controlling confounding factors since it aims to homogenize the basal conditions of the participants between the groups, reducing the probability that the observed association is due to a variable other than the intervention[9]. In addition, its prospective character meets the criteria of temporality, given that the exposure precedes the effect with certainty, which supports a causal effect[8]. Randomized clinical trials are considered "the cornerstone" of evidence-based healthcare, as they are the central component of systematic reviews devoted to answering research questions associated with interventions. Since systematic reviews integrate the results of multiple randomized clinical trials, they refine the estimate of the effect size and provide a higher level of scientific evidence for the evaluation of health claims compared to a single randomized clinical trial. The hierarchy of evidence in health sciences is depicted as a pyramid (Figure 1)[10].

Figure 1. The classical conceptualization of the hierarchy of health evidence

From association to causation: from the preclinical model to the human organism

In 1965, the British epidemiologist Austin Bradford Hill proposed the "principle of consistency" in the context of nine aspects of association that respond the question: "In what circumstances can we pass from this observed association to a verdict of causation?"[11]. These aspects, later known as the Bradford Hill Criteria[12], confer higher probability that the association observed between two factors are causal. The principle of coherence manifests that a causal conclusion should not contradict contemporary foundations in knowledge[12]. While this concept is a matter for discussion and debate, it can be understood as a need for congruence between evidence from preclinical studies (in vitro, in vivo) and clinical studies. Without correlation, a causal relationship might be questioned.

A good example is that of phenolic compounds in foods. Despite the fact that multiple potentially beneficial effects of different molecules of this type have been demonstrated in in vitro assays and animal models, attributing antioxidant, anti-inflammatory, anticarcinogenic, cardioprotective and neuroprotective properties, among others, these effects have not consistently been observed in clinical studies[13]. To date, only two health claims for polyphenols have been substantiated through human studies and approved by regulatory agencies—hydroxytyrosol of olive oil[14] and cocoa flavanols[15]. Thus, the lack of consistency between laboratory assays and clinical studies precludes the scientific substantiation of the physiological effects of these compounds[16],[17].

Currently, to consider a causal inference as true, research must integrate the findings of multiple scientific disciplines[18]. Consequently, it is essential that human studies incorporate methodological designs that examine the effectiveness of an intervention and can establish causality; as stated above, this can be achieved by randomized clinical trials and with greater robustness by systematic reviews of randomized clinical trials. By way of example, the Ministry of Health, Labor and Well-being of Japan explicitly states that to validate the effectiveness of a product two methods may be used to evidence the purported effects of a product or specific components: randomized clinical trials or systematic reviews of randomized clinical trials (http://www.caa.go.jp/foods/index23.html).

Systematic reviews of the literature

Systematic reviews constitute a method widely used in biomedical research applied to the pharmacological effects, among other therapeutic interventions aimed at improving health. This tool is also very useful for evaluating scientific evidence related to the physiological effects of the intake of bioactive food components. Systematic reviews, using the standardized and reproducible methodology outlined in an a priori protocol makes it possible to answer the question whether a specific physiological or clinical effect is produced[19]. Systematic reviews follow several stages [20],[21],[22],[23],[24]. Initially, a hypothesis is set forth with clear objectives around a research question. A comprehensive search of primary studies (randomized clinical trials) is performed in multiple relevant databases, as well as in unpublished literature (“gray literature”), such as congress abstracts or other documents[25]. Randomized clinical trials that answer the study question are identified, and then selected according to pre-established criteria[26]. For example, studies must be consistent between the evaluated product concerning consumption habits and the food matrix. Besides including trials with a well-defined intervention, other considerations include similar, well-characterized populations, and well-defined outcomes of interest (biomarker or clinical effect)[27],[28]. Subsequent stages include assessment of the methodological quality and analysis of the risk of bias for each study using standardized tools[29], data extraction from primary studies, synthesis of the collected evidence and interpretation of the findings.

It is possible to perform a statistical quantitative analysis called meta-analysis, in which the estimates from individual studies are synthesized into a combined estimate, so long as the heterogeneity of the data permits it[30]. GRADE methodology (Grades of Recommendation, Assessment, Development and Evaluation) is applied to formulate the conclusions, and generates a Summary of Findings table (http://www.gradeworkinggroup.org/) to present the results, together with a classification of the certainty of the totality of the evidence based on the quality of the individual studies included. The GRADE score will be lowered if the body of evidence contains low quality randomized clinical trials. On the other hand, if the included observational studies have controlled important biases, their grading will be increased.

Murad et al.[31]  have proposed a new conceptualization of the traditional hierarchy of evidence (Figure 1), recognizing the role of systematic reviews in integrating the existing evidence, like a magnifying glass through which evidence is evaluated (Figure 2). Additionally, this conceptualization recognizes the role of GRADE methodology, since it presents wavy lines between methodological designs in the hierarchy, reflecting that the quality of the evidence shows fluctuating boundaries according to the included studies.

Figure 2. New conceptualization of the hierarchy of evidence.

Final considerations

Systematic reviews are a very useful tool in the validation of physiological effects of foods or food constituents, enabling to substantiate their functional properties scientifically. They are based on the results of randomized clinical trials, stringent in their design, and make it possible to establish a causal association between the food and the purported physiological effect. If the food or some of its constituents exhibit effects only in laboratory tests, in vitro assays or animal models, but  randomized clinical trials are not consistent or have not been performed, as is the case for many bioactive compounds (polyphenols, carotenoids, sulfur compounds, among others), it is not possible to substantiate their effects and call them as functional foods. This lack of coherence is reflected in the small number of health claims approved by regulatory agencies internationally, in contrast to the high demand for approval.

The recognition of systematic reviews in the scientific substantiation of health claims in the field of foods is increasingly recognized by those interested in the topic, not only in academia and in research, but also by industry, regulatory agencies, and consumers. Systematic reviews provide relevant information to decision makers and are a fundamental tool for the truthful communication of health properties of foods or components to consumers, preventing confusion, misinformation, and deception.

Notes

Roles and contributions of the authors
ML: conceptualization, methodology, research, writing (preparation of the original draft), writing (revision and edition), visualization, supervision, administration of the project.
MA: conceptualization, methodology, research, writing (preparation of the original draft), writing (revision and edition), visualization, supervision, administration of the project.
JS: conceptualization, methodology, research, writing (preparation of the original draft), writing (revision and edition), visualization, supervision, administration of the project.

Funding
Authors declare that there was no source of funding.

Conflicts of interest
The authors completed the Declaration of Conflicts of Interest of ICMJE and declared they did not receive funds associated with this article; they do not have financial relationships with organizations that may have an interest in the article over the last three years and have no other relationships or activities that may influence on the publication of the article. Forms are available by contacting the corresponding author or the Editorial Committee of the Journal.

From the editors
The original version of this manuscript was submitted in Spanish. The authors provided a translation that was lightly edited by the journal.

Figure 1. The classical conceptualization of the hierarchy of health evidence
Figure 2. New conceptualization of the hierarchy of evidence.
Licencia Creative Commons Esta obra de Medwave está bajo una licencia Creative Commons Atribución-NoComercial 3.0 Unported. Esta licencia permite el uso, distribución y reproducción del artículo en cualquier medio, siempre y cuando se otorgue el crédito correspondiente al autor del artículo y al medio en que se publica, en este caso, Medwave.

 

Systematic literature reviews are one of the main methodologies used to substantiate the health properties of foods and food constituents purported to affect human physiology. This tool is based on scientific evidence obtained from correctly performed randomized controlled trials. Systematic reviews make it possible to conclude whether there is a causal relationship between food consumed and health effects observed, thus supporting the use of the term “functional foods.” We present and analyze the prinicpal characteristics of, and ways by which, systematic reviews can contribute to the regulatory approval of health claims directed to consumers.

Autores: Mariane Lutz[1], Marcelo Arancibia[1], Jana Stojanova[1]

Filiación:
[1] Centro Interdisciplinario de Estudios en Salud (CIESAL), Escuela de Medicina, Universidad de Valparaíso, Viña del Mar, Chile

E-mail: marcelo.arancibiame@uv.cl

Correspondencia a:
[1] Angamos 655
Campus de la Salud
Oficina 1107
Reñaca
Viña del Mar
Chile

Citación: Lutz M, Arancibia M, Stojanova J. Using systematic reviews in the scientific substantiation of health properties of foods and food constituents. Medwave 2019;19(6):e7664 doi: 10.5867/medwave.2019.06.7664

Fecha de envío: 25/5/2019

Fecha de aceptación: 25/6/2019

Fecha de publicación: 11/7/2019

Origen: no solicitado

Tipo de revisión: con revisión por pares externa, por tres pares revisores a doble ciego

Ficha PubMed

Comentarios (0)

Nos complace que usted tenga interés en comentar uno de nuestros artículos. Su comentario será publicado inmediatamente. No obstante, Medwave se reserva el derecho a eliminarlo posteriormente si la dirección editorial considera que su comentario es: ofensivo en algún sentido, irrelevante, trivial, contiene errores de lenguaje, contiene arengas políticas, obedece a fines comerciales, contiene datos de alguna persona en particular, o sugiere cambios en el manejo de pacientes que no hayan sido publicados previamente en alguna revista con revisión por pares.

Aún no hay comentarios en este artículo.


Para comentar debe iniciar sesión

Medwave publica las vistas HTML y descargas PDF por artículo, junto con otras métricas de redes sociales.

Se puede producir un retraso de 48 horas en la actualización de las estadísticas.

  1. Lutz M. ¿Podemos Hablar De Alimentos Funcionales En Chile? Rev Chil Nutr.2012 Jun;39(2):211-6. | Link |
  2. Lutz M. Science behind the substantiation of health claims in functional foods: current regulations. In: Functional Foods and Biotechnology. CRC Press; 2019.
  3. Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993 Dec 31;703:125-33; discussion 133-4. | CrossRef | PubMed |
  4. Mann JI. Evidence-based nutrition: Does it differ from evidence-based medicine? Ann Med. 2010 Oct;42(7):475-86. | CrossRef | PubMed |
  5. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002 Jan 19;359(9302):248-52. | PubMed |
  6. Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009 May;63(5):691-7. | CrossRef | PubMed |
  7. Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet. 2002 Jan 5;359(9300):57-61. | PubMed |
  8. Spieth PM, Kubasch AS, Penzlin AI, Illigens BM, Barlinn K, Siepmann T. Randomized controlled trials - a matter of design. Neuropsychiatr Dis Treat. 2016 Jun 10;12:1341-9. | CrossRef | PubMed |
  9. Suresh K. An overview of randomization techniques: An unbiased assessment of outcome in clinical research. J Hum Reprod Sci. 2011 Jan;4(1):8-11. | CrossRef | PubMed |
  10. Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid hierarchy? Circulation. 2008 Oct 14;118(16):1675-84. | CrossRef | PubMed |
  11. Hill A. The environment and disease: association or causation? Proc R Soc Med. 1965 May;58(5):295–300. | PubMed |
  12. Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005 Nov 3;2:11. | PubMed |
  13. Lutz M, Fuentes E, Ávila F, Alarcón M, Palomo I. Roles of Phenolic Compounds in the Reduction of Risk Factors of Cardiovascular Diseases. Molecules. 2019 Jan 21;24(2). pii: E366. | CrossRef | PubMed |
  14. Scientific Opinion on the substantiation of health claims related to polyphenols in olive and protection of LDL particles from oxidative damage (ID 1333, 1638, 1639, 1696, 2865), maintenance of normal blood HDL cholesterol concentrations (ID 1639), mainte. EFSA J. 2011 Apr 1;9(4):2033. | Link |
  15. Scientific Opinion on the substantiation of a health claim related to cocoa flavanols and maintenance of normal endothelium‐dependent vasodilation pursuant to Article 13(5) of Regulation (EC) No 1924/2006. EFSA J. 2012 Jul;10(7). | Link |
  16. Ávila-Gálvez MÁ, González-Sarrías A, Espín JC. In Vitro Research on Dietary Polyphenols and Health: A Call of Caution and a Guide on How To Proceed. J Agric Food Chem. 2018 Aug 1;66(30):7857-7858. | CrossRef | PubMed |
  17. Cory H, Passarelli S, Szeto J, Tamez M, Mattei J. The Role of Polyphenols in Human Health and Food Systems: A Mini-Review. Front Nutr. 2018 Sep 21;5:87. | CrossRef | PubMed |
  18. Fedak KM, Bernal A, Capshaw ZA, Gross S. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015 Sep 30;12:14. | CrossRef | PubMed |
  19. Mittal N, Goyal M, Mittal PK. Understanding and Appraising Systematic Reviews and Meta-Analysis. J Clin Pediatr Dent. 2017;41(5):317-326. | CrossRef | PubMed |
  20. Glasziou P, Irwig L, Bain C, Colditz G. Systematic Reviews in Health Care. Cambridge: Cambridge University Press; 2001. | Link |
  21. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. | Link |
  22. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009 Jul 21;339:b2700. | CrossRef | PubMed |
  23. Gough G, Oliver S, Thomas J. An Introduction to Systematic Reviews. London: SAGE Publications; 2012.
  24. Khan K, Zunz R, Kleijnen J, Antes G. Systematic Reviews to Support Evidence-based Medicine. London: Royal Society of Medicine Press; 2003. | Link |
  25. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000010. | PubMed |
  26. Buckley DI, Ansari MT, Butler M, Soh C, Chang CS. The refinement of topics for systematic reviews: lessons and recommendations from the Effective Health Care Program. J Clin Epidemiol. 2014 Apr;67(4):425-32. | CrossRef | PubMed |
  27. Franco JVA, Arancibia M, Simancas-Racines D, Madrid E. Syntheses of biomedical information: narrative reviews, systematic reviews and emerging formats. Medwave. 2018 Nov 27;18(7):e7354. | CrossRef | PubMed |
  28. Pae C-U. Why Systematic Review rather than Narrative Review? Psychiatry Investig. 2015 Jul;12(3):417–9. | CrossRef | PubMed |
  29. Handu D, Moloney L, Wolfram T, Ziegler P, Acosta A, Steiber A. Academy of Nutrition and Dietetics Methodology for Conducting Systematic Reviews for the Evidence Analysis Library. J Acad Nutr Diet. 2016 Feb;116(2):311-8. | CrossRef | PubMed |
  30. Higgins J, Thompson S, Deeks J, Altman D. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002 Jan;7(1):51-61. | PubMed |
  31. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016 Aug;21(4):125-7. | CrossRef | PubMed |
Lutz M. ¿Podemos Hablar De Alimentos Funcionales En Chile? Rev Chil Nutr.2012 Jun;39(2):211-6. | Link |

Lutz M. Science behind the substantiation of health claims in functional foods: current regulations. In: Functional Foods and Biotechnology. CRC Press; 2019.

Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993 Dec 31;703:125-33; discussion 133-4. | CrossRef | PubMed |

Mann JI. Evidence-based nutrition: Does it differ from evidence-based medicine? Ann Med. 2010 Oct;42(7):475-86. | CrossRef | PubMed |

Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002 Jan 19;359(9302):248-52. | PubMed |

Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009 May;63(5):691-7. | CrossRef | PubMed |

Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet. 2002 Jan 5;359(9300):57-61. | PubMed |

Spieth PM, Kubasch AS, Penzlin AI, Illigens BM, Barlinn K, Siepmann T. Randomized controlled trials - a matter of design. Neuropsychiatr Dis Treat. 2016 Jun 10;12:1341-9. | CrossRef | PubMed |

Suresh K. An overview of randomization techniques: An unbiased assessment of outcome in clinical research. J Hum Reprod Sci. 2011 Jan;4(1):8-11. | CrossRef | PubMed |

Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid hierarchy? Circulation. 2008 Oct 14;118(16):1675-84. | CrossRef | PubMed |

Hill A. The environment and disease: association or causation? Proc R Soc Med. 1965 May;58(5):295–300. | PubMed |

Höfler M. The Bradford Hill considerations on causality: a counterfactual perspective. Emerg Themes Epidemiol. 2005 Nov 3;2:11. | PubMed |

Lutz M, Fuentes E, Ávila F, Alarcón M, Palomo I. Roles of Phenolic Compounds in the Reduction of Risk Factors of Cardiovascular Diseases. Molecules. 2019 Jan 21;24(2). pii: E366. | CrossRef | PubMed |

Scientific Opinion on the substantiation of health claims related to polyphenols in olive and protection of LDL particles from oxidative damage (ID 1333, 1638, 1639, 1696, 2865), maintenance of normal blood HDL cholesterol concentrations (ID 1639), mainte. EFSA J. 2011 Apr 1;9(4):2033. | Link |

Scientific Opinion on the substantiation of a health claim related to cocoa flavanols and maintenance of normal endothelium‐dependent vasodilation pursuant to Article 13(5) of Regulation (EC) No 1924/2006. EFSA J. 2012 Jul;10(7). | Link |

Ávila-Gálvez MÁ, González-Sarrías A, Espín JC. In Vitro Research on Dietary Polyphenols and Health: A Call of Caution and a Guide on How To Proceed. J Agric Food Chem. 2018 Aug 1;66(30):7857-7858. | CrossRef | PubMed |

Cory H, Passarelli S, Szeto J, Tamez M, Mattei J. The Role of Polyphenols in Human Health and Food Systems: A Mini-Review. Front Nutr. 2018 Sep 21;5:87. | CrossRef | PubMed |

Fedak KM, Bernal A, Capshaw ZA, Gross S. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015 Sep 30;12:14. | CrossRef | PubMed |

Mittal N, Goyal M, Mittal PK. Understanding and Appraising Systematic Reviews and Meta-Analysis. J Clin Pediatr Dent. 2017;41(5):317-326. | CrossRef | PubMed |

Glasziou P, Irwig L, Bain C, Colditz G. Systematic Reviews in Health Care. Cambridge: Cambridge University Press; 2001. | Link |

Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. | Link |

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009 Jul 21;339:b2700. | CrossRef | PubMed |

Gough G, Oliver S, Thomas J. An Introduction to Systematic Reviews. London: SAGE Publications; 2012.

Khan K, Zunz R, Kleijnen J, Antes G. Systematic Reviews to Support Evidence-based Medicine. London: Royal Society of Medicine Press; 2003. | Link |

Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000010. | PubMed |

Buckley DI, Ansari MT, Butler M, Soh C, Chang CS. The refinement of topics for systematic reviews: lessons and recommendations from the Effective Health Care Program. J Clin Epidemiol. 2014 Apr;67(4):425-32. | CrossRef | PubMed |

Franco JVA, Arancibia M, Simancas-Racines D, Madrid E. Syntheses of biomedical information: narrative reviews, systematic reviews and emerging formats. Medwave. 2018 Nov 27;18(7):e7354. | CrossRef | PubMed |

Pae C-U. Why Systematic Review rather than Narrative Review? Psychiatry Investig. 2015 Jul;12(3):417–9. | CrossRef | PubMed |

Handu D, Moloney L, Wolfram T, Ziegler P, Acosta A, Steiber A. Academy of Nutrition and Dietetics Methodology for Conducting Systematic Reviews for the Evidence Analysis Library. J Acad Nutr Diet. 2016 Feb;116(2):311-8. | CrossRef | PubMed |

Higgins J, Thompson S, Deeks J, Altman D. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002 Jan;7(1):51-61. | PubMed |

Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016 Aug;21(4):125-7. | CrossRef | PubMed |