Key Words: GRADE approach, practice guideline, health equity
The GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology provides a framework for assessing the certainty of the evidence and making recommendations. The Evidence to Decision Framework (EtD) is a transparent and structured system for formulating health recommendations. Once the problem is identified and the certainty of the evidence is assessed, EtD provides several criteria for formulating a recommendation. These criteria include the trade-off between benefits and harms, patients’ values and preferences, acceptability, feasibility, resource use, and impact on equity. The resulting recommendations may differ in strength (strong or weak) and direction (for or against). The process is transparent, allowing other users to adjust the framework of recommendations by modifying the criteria to fit the desired context through an adaptation-adoption process. Given the extensive information available on EtD and the GRADE methodology in general, this narrative review seeks to explain the main concepts involved in decision-making in health by using simplified and friendly descriptions, accompanied by practical examples, thus facilitating its understanding by inexperienced readers.
This article aims to provide a simplified explanation of the Evidence to Decision (EtD) frameworks to novice readers, as was done in the first part in which the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach to assessing the certainty of evidence was developed. In this second part, the EtD frameworks and their general concepts will be addressed, without particular emphasis on more complex scenarios, detailed at length in the GRADE Handbook itself and other bibliographic resources. EtD frameworks emerge to integrate the three key components of evidence-based medicine in an efficient, systematized, and structured way: best available evidence, clinical expertise/experience, and patient values and preferences. In turn, they seek to standardize, based on transparent and explicit frameworks, how to develop recommendations for health-care decision-making, whether for prevention, diagnosis, or therapy. Should patients with sepsis receive early antibiotics? Should we screen for prostate cancer with prostate-specific antigen? Should we implement a breast cancer screening program with mammography? To make recommendations on questions such as these, one must not only evaluate the best available evidence but also consider additional factors that will be developed throughout this article.
It is important to mention that, although these recommendations can be generated de novo, due to the transparency of the process, they can also be adapted/adopted from others already developed, adapting the criteria to the desired context through an adaptation–adoption process according to the GRADE methodology.
This article is the fourth in a methodological series of narrative reviews on general biostatistics and clinical epidemiology topics, which explore and summarize in user-friendly language published articles available in the main databases and specialized reference texts. The series is oriented to the training of undergraduate and graduate students. It is carried out by the Chair of Evidence-Based Medicine of the School of Medicine of the University of Valparaiso, Chile, in collaboration with the Research Department of the University Institute of the Italian Hospital of Buenos Aires, Argentina, and the UC Evidence Center of the Catholic University, Chile. This article aims to describe the EtD frameworks, a component of the GRADE methodology useful to homogenize and improve healthcare decision-making.
Broadly speaking, the GRADE methodology consists of two main processes: the assessment of the certainty of the evidence and the generation of recommendations. The first is to clearly formulate the question in PICO format (patient/problem, intervention, comparison, and outcomes), considering the different perspectives (relevant subgroups and important context for understanding the question) and why a recommendation or decision is necessary. Subsequently, the evidence for outcomes should be evaluated according to their corresponding studies (e.g., whether they are interventions in clinical trials or observational studies); and synthesize this evidence, ideally in systematic reviews, using the GRADE approach to assess their certainty. This process has been described in the first part of this installment. In a second instance, EtD frameworks seek to formulate recommendations in a structured and transparent manner, considering explicit criteria that support why a particular health decision is proposed. Figure 1 summarizes the complete process of applying the GRADE methodology, from the assessment of the certainty of the evidence to the application of the EtD frameworks.
EtD frameworks are schematic and transparent frameworks for formulating health recommendations, which show, using explicit criteria, the reasons for generating such recommendations. In addition to the certainty of the evidence, they consider the balance between benefits and harms, patients’ values and preferences, and more concrete aspects such as acceptability, feasibility, and costs. The process of applying these criteria for the formation of recommendations is shown in a simplified manner in Figure 2.
The first two criteria (A and B) are closely related because the balance depends on the evidence of benefits and harms, as well as the relative importance of each, defined by the values and preferences of patients. The following two (C and D) are focused on the practical application of the recommendations.
A. The balance between benefits and harms
The GRADE approach evaluates the certainty of the evidence for each of the outcomes (e.g., mortality or quality of life, with their respective certainty). However, to determine the net balance between benefits and harms, several outcomes must be evaluated simultaneously. Therefore, it is important to know the certainty of the evidence for each outcome of benefit or harm and the weight of each outcome in decision-making, according to the perspectives of individuals. The evaluation of this balance between benefits and harms, as well as the other criteria of the EtD framework, can indicate the strength and direction of the recommendation. For example, the early use of antibiotics in sepsis is based on a net benefit balance for patients suffering from this condition (Example 1).
Example 1. The balance between benefits and harms: use of antibiotics in sepsis
Sepsis is a serious medical condition characterized by an inflammatory response to an infection that can affect the entire body, with a very high mortality rate without treatment. Most patients with sepsis require antibiotics and admission to an intensive care unit. In this context, the aim is to reduce the mortality outcome in patients with sepsis. The administration of antibiotics far outweighs the benefit/harm balance, with the adverse effects of antibiotics playing a lesser role in this balance, regardless of the exact time of antibiotic application.
B. People’s values and preferences
The values and preferences of individuals are defined as the relative importance of the outcomes, that is, the value that each individual places on each outcome from his or her perspective. These can be determined directly in the clinical encounter with patients, involving people in making decisions about their health, or developing clinical guidelines, using the available evidence on preferences. The evidence on preference, in turn, can have different levels of certainty, for which the GRADE approach has developed a specific methodology,.
Systematic study of people’s values and preferences is scarce, and evidence is often not available. However, clinical experience can provide considerable information about typical values and preferences. An example is the knowledge of the strong aversion of pregnant women to the risk of major fetal anomalies when receiving a drug, even when the associated risk is considered small. However, there may be evidence reporting large variability or discordance among individuals’ preferences, and a single recommendation is less likely to apply uniformly to all patients. Therefore, when there is large uncertainty or variability, it is impossible to make a strong recommendation for or against, as this would not apply to the entire target population (Example 2, regarding prostate cancer screening).
Example 2. Weak recommendation: prostate cancer screening according to net balance values and preferences
Prostate-specific antigen screening for prostate cancer screening slightly decreases prostate-specific mortality. However, it causes harm through false positives, overdiagnosis, and overtreatment. For example, treatment of indolent prostate cancers leaves sequelae in urinary and sexual function. Some men may value more the effect on specific mortality, and others may prefer preservation of urinary and sexual function. Considering the magnitude of harms evidenced by this practice and the variability in values and preferences, a weak recommendation against it has been made that clinicians should not routinely consider screening except in specific cases that will require shared decision-making to respect individual preferences.
C. Acceptability and feasibility
An intervention considered beneficial may not be acceptable for various reasons, whether personal, community, or religious; or it may be considered unfeasible because it is incompatible with the local reality for which it was originally intended (Examples 3a and 3b).
Example 3a. Acceptability: the case of blood transfusions
In most cases, a blood transfusion can be a beneficial intervention (with a positive net balance). However, respecting individual autonomy, a person may not accept it on religious grounds, despite recognizing that the balance of benefits and harms is favorable. By considering the acceptability of an intervention, possible barriers to its implementation are anticipated and allow, eventually, to adapt the recommendation for subgroups of people (e.g., by means of autologous blood transfusion).
Example 3b. Feasibility: the case of new technologies
The use of electronic devices can be associated with substantial health benefits in certain scenarios. However, they depend on contingent factors such as telephone and data networks. In turn, an intervention involving the use of electronic devices aimed at people with low digital literacy (e.g., some groups of older adults) will be considered difficult to implement, even if the benefits are clear.
D. Resource use, equity, and cost-effectiveness
There are circumstances in which resource use is a key factor in the development of a recommendation, especially when the balance between benefits and harms is narrow or the implementation of the recommendation has important budgetary implications (Example 4). Similar to values and preferences, there is often little information on the economic implications of recommendations, especially considering the direct and indirect costs involved. The EtD frameworks indicate that some key factors should be considered in estimating resources for the formulation of recommendations:
a) The funder’s perspective in estimating costs.
b) The funder’s perspective on resource use.
c) Local (jurisdictional) variations in cost estimation.
d) The perspective in defining the opportunity cost (what is left out to finance the implementation of the recommendation).
e) Political implications in the allocation of resources.
There are two tools of health economics that allow this criterion to be assessed:
Finally, when resources are used for one intervention, they are not used for others (opportunity cost), potentially increasing inequalities between individuals, affecting equity. For example, suppose there is a highly effective or moderately effective drug for a group of people within a country whose target population is small, and the drug is expensive. In that case, this intervention can displace the budget allocated to measures with a greater impact on society (vaccines and water sanitation), increasing social gaps by affecting more vulnerable groups.
Example 4. Application of criterion D: mammographic screening
In Argentina, the National Cancer Institute used the decision framework questionnaire to establish recommendations on whether or not to perform mammographic screening from a public health perspective, separated by age group.
For women aged 40 to 49 years, the expert panel determined that the resource costs required were extensive (low certainty of evidence), and the cost-effectiveness of the intervention likely benefits the comparison (i.e., do not perform). Therefore, the direction of the intervention recommendation was against screening (strength of recommendation: conditional). Meanwhile, for women aged 50 to 69 years, although the panel considered the resources required involve extensive costs (moderate certainty of evidence), the cost-effectiveness of the intervention favors the intervention. Consequently, the direction of the intervention recommendation was in favor (strength of recommendation: strong).
Acceptability, feasibility, and cost-effectiveness depend on efficacy. An intervention will be more acceptable if this same intervention is very effective. In turn, if the intervention is very effective, the cost-effectiveness ratio will be better. There is interdependence between the evidence and the criteria or domains to be considered, which justifies their separation within the EtD framework.
The recommendations formulated by the EtD frameworks can be classified according to their strength (strong or weak) and according to their direction (for or against). To summarize, the strength of a recommendation reflects the degree of confidence that the desired effects of an intervention outweigh the undesirable effects or vice versa. In strong recommendations, the balance between benefits/harm, considering also costs, and patient values and preferences, is clearly determined. In weak recommendations, the opposite is true. It is important to clarify that the term “weak recommendation” can be interpreted negatively by associating the word weak with weak evidence. In order to avoid misunderstandings, the GRADE group proposes to describe weak recommendations with other terms, such as conditional and discretionary. If any of these variations are used, authors must be consistent throughout their guidelines and the rest of the guidelines they produce.
It should be noted that even when confidence in the estimate of effect is low and/or the desirable and undesirable consequences are closely balanced, GRADE encourages expert panels to overcome doubts and formulate recommendations, which will inevitably be weak. Table 1 summarizes the significance of these recommendations for patients, clinicians, and policymakers.
If we take the case of the strong recommendation in favor of early antibiotic use in patients with sepsis (Example 1), it is expected that most patients will receive this medication and that most (if not all) clinicians will take this course of action. Indeed, the proportion of patients with sepsis receiving antibiotics early could be a definite quality-of-care indicator for hospital policy developers.
In contrast, in the case of the weak recommendation against prostate cancer screening (Example 2), there is variability in patient values and preferences, so the decision is expected to be individualized. In this case, decision support tools (decision aids) could be incorporated to assist the dialogue with patients, with the use of such tools for communication being a possible quality indicator. The proportion of men screened with PSA could not be an indicator of quality because variable implementation is reasonably anticipated.
Recommendations should not be regarded as rules or rulings. Even strong recommendations based on evidence of high certainty will not apply to all circumstances or all patients. Guideline users may reasonably conclude that following particular recommendations based on high certainty of evidence will be wrong for some patients considering their unique characteristics.
Strength and direction of recommendations
There is no direct relationship between the certainty of evidence and the strength of recommendation. The certainty of the evidence is assessed for each outcome; usually, several outcomes are estimated for an intervention, and in addition, all the other criteria already mentioned should be considered. For example, an intervention with high certainty of evidence for its effectiveness may end up with a recommendation against it due to uncertainty about its safety (about its adverse effects). An example of this is the use of quinolones to treat urinary tract infections during pregnancy. Conversely, an intervention for which there is low certainty of evidence for its effectiveness may end up with a strong recommendation in favor of, for example, the duration of handwashing.
There are many recommendations on prevention, diagnosis, and treatment of health or public health problems developed by teams that systematically apply this methodology. To avoid unnecessary efforts, the GRADE group created the adaptation–adoption frameworks, making it possible to adapt or adopt recommendations made in other countries. For example, in Argentina, the breast cancer screening recommendation (Example 4) was developed from other current recommendations that used the GRADE method using the adaptation–adoption process developed for this purpose (Figure 3).
Initially, we search for relevant guidelines with GRADE methodology, and we can update the evidence that supports them. Then the EtD framework is reassessed in the local context. If the recommendation is similar, it is called “adoption”, if it is different, “adaptation”, and in the case of not having GRADE guidelines, one can choose to develop new recommendations from scratch or “de novo”.
The available evidence on the effectiveness of interventions should be considered together with evidence of other types, mainly people’s values and preferences. Specific aspects of the applicability of an intervention, such as acceptability, feasibility, and costs, should also be included. As this process is developed transparently, the EtD frameworks are adoptable and adaptable by other territories.
The certainty of the evidence is not directly proportional to the strength of the recommendation, given the multiple facets of health problems.
As in the first part of our description of the GRADE method, using simple descriptions accompanied by examples, we seek to provide a simplified and summarized approach to this methodology for its understanding and application by the general public, students, and novice methodologists.
CQ: conceptualization, methodology, research, writing original draft, review and editing, visualization, project management. MK, BV: conceptualization, methodology, research, writing original draft, reviewing and editing, visualization. JPB, LG: conceptualization, methodology, review and editing, visualization. JAF: conceptualization, methodology, writing original draft, review and editing, visualization.
We thank the Chair of Evidence-Based Medicine of the School of Medicine of the University of Valparaíso, Chile, for promoting this series and the collaboration of the Research Department of the University Institute of the Italian Hospital of Buenos Aires, Argentina.
The authors completed the ICMJE conflict of interest statement and declared that they did not receive funding for this article; they have no financial relationships with organizations that may have an interest in the article published in the last three years and they have no other relationships or activities that may influence the publication of the article. Forms can be requested by contacting the responsible author or the Editorial Board of the Journal.
The authors declare that they have no external sources of financing associated with the preparation of this article.
Language of submission
La metodología GRADE (Grading of Recommendations Assessment, Development and Evaluation) provee un método para valorar la certeza de la evidencia y para la formulación de recomendaciones que ayuden en la toma de decisiones en salud. Una vez identificado el problema y valorada la certeza de la evidencia, los marcos de la evidencia a la decisión (Evidence to decision frameworks o EtD) utilizan un sistema transparente y estructurado para formular recomendaciones, en que se consideran criterios como el balance entre beneficios y daños, valores y preferencias de los pacientes, aceptabilidad, factibilidad, equidad y uso de recursos para establecer una recomendación en salud. Esta podrá tener distinta fuerza (fuertes o débiles) y dirección (a favor o en contra). La transparencia de este proceso permite que otros usuarios puedan adaptar un marco de recomendaciones según el contexto deseado, mediante un proceso de adaptación-adopción. Ante la extensa información disponible sobre marcos de la evidencia a la decisión y la metodología GRADE en general, esta revisión narrativa busca entregar una explicación acerca de los principales conceptos involucrados en la toma de decisiones en salud, a través de descripciones simplificadas, amigables y con ejemplos prácticos, con el fin de facilitar su entendimiento por lectores inexpertos.
Citation: Quilodrán C, Kirmayr M, Valente B, Pérez-Bracchiglione J, Garegnani L, Franco JVA. The GRADE approach, Part 2: Evidence to decision frameworks—outlining decision-making in health. Medwave 2021;21(4):e8182 doi: 10.5867/medwave.2021.04.8182
Submission date: 22/9/2020
Acceptance date: 15/4/2021
Publication date: 7/5/2021
Origin: Not commissioned
Type of review: Externally peer-reviewed by three reviewers, double-blind
We are pleased to have your comment on one of our articles. Your comment will be published as soon as it is posted. However, Medwave reserves the right to remove it later if the editors consider your comment to be: offensive in some sense, irrelevant, trivial, contains grammatical mistakes, contains political harangues, appears to be advertising, contains data from a particular person or suggests the need for changes in practice in terms of diagnostic, preventive or therapeutic interventions, if that evidence has not previously been published in a peer-reviewed journal.
No comments on this article.
To comment please log in