Introduction

This article aims to provide a simplified explanation of the Evidence to Decision (EtD) frameworks to novice readers, as was done in the first part[1] in which the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach to assessing the certainty of evidence was developed. In this second part, the EtD frameworks and their general concepts will be addressed, without particular emphasis on more complex scenarios, detailed at length in the GRADE Handbook itself[2] and other bibliographic resources. EtD frameworks emerge to integrate the three key components of evidence-based medicine in an efficient, systematized, and structured way: best available evidence, clinical expertise/experience, and patient values and preferences[3]. In turn, they seek to standardize, based on transparent and explicit frameworks, how to develop recommendations for health-care decision-making, whether for prevention, diagnosis, or therapy[3]. Should patients with sepsis receive early antibiotics? Should we screen for prostate cancer with prostate-specific antigen? Should we implement a breast cancer screening program with mammography? To make recommendations on questions such as these, one must not only evaluate the best available evidence but also consider additional factors that will be developed throughout this article.

It is important to mention that, although these recommendations can be generated de novo, due to the transparency of the process, they can also be adapted/adopted from others already developed, adapting the criteria to the desired context through an adaptation–adoption process according to the GRADE methodology[4].

This article is the fourth in a methodological series of narrative reviews on general biostatistics and clinical epidemiology topics, which explore and summarize in user-friendly language published articles available in the main databases and specialized reference texts. The series is oriented to the training of undergraduate and graduate students. It is carried out by the Chair of Evidence-Based Medicine of the School of Medicine of the University of Valparaiso, Chile, in collaboration with the Research Department of the University Institute of the Italian Hospital of Buenos Aires, Argentina, and the UC Evidence Center of the Catholic University, Chile. This article aims to describe the EtD frameworks, a component of the GRADE methodology useful to homogenize and improve healthcare decision-making.

Overview of the GRADE methodology

Broadly speaking, the GRADE methodology consists of two main processes: the assessment of the certainty of the evidence and the generation of recommendations. The first is to clearly formulate the question in PICO format (patient/problem, intervention, comparison, and outcomes), considering the different perspectives (relevant subgroups and important context for understanding the question) and why a recommendation or decision is necessary[5]. Subsequently, the evidence for outcomes should be evaluated according to their corresponding studies (e.g., whether they are interventions in clinical trials or observational studies); and synthesize this evidence, ideally in systematic reviews, using the GRADE approach to assess their certainty[6]. This process has been described in the first part of this installment[1]. In a second instance, EtD frameworks seek to formulate recommendations in a structured and transparent manner, considering explicit criteria that support why a particular health decision is proposed. Figure 1 summarizes the complete process of applying the GRADE methodology, from the assessment of the certainty of the evidence to the application of the EtD frameworks.

Figure 1.

Summary of the GRADE methodology.

Full size

Evidence to Decision frameworks and their structure

EtD frameworks are schematic and transparent frameworks for formulating health recommendations, which show, using explicit criteria, the reasons for generating such recommendations[4]. In addition to the certainty of the evidence, they consider the balance between benefits and harms, patients’ values and preferences, and more concrete aspects such as acceptability, feasibility, and costs. The process of applying these criteria for the formation of recommendations is shown in a simplified manner in Figure 2.

Figure 2.

Graphical summary of the recommendation formulation process proposed by the EtDs.

Full size

The first two criteria (A and B) are closely related because the balance depends on the evidence of benefits and harms, as well as the relative importance of each, defined by the values and preferences of patients. The following two (C and D) are focused on the practical application of the recommendations.

Application of the criteria in the evidence frameworks to the decision

A. The balance between benefits and harms

The GRADE approach evaluates the certainty of the evidence for each of the outcomes (e.g., mortality or quality of life, with their respective certainty). However, to determine the net balance between benefits and harms, several outcomes must be evaluated simultaneously. Therefore, it is important to know the certainty of the evidence for each outcome of benefit or harm and the weight of each outcome in decision-making, according to the perspectives of individuals[8]. The evaluation of this balance between benefits and harms, as well as the other criteria of the EtD framework, can indicate the strength and direction of the recommendation. For example, the early use of antibiotics in sepsis is based on a net benefit balance for patients suffering from this condition (Example 1).

B. People’s values and preferences

The values and preferences of individuals are defined as the relative importance of the outcomes, that is, the value that each individual places on each outcome from his or her perspective. These can be determined directly in the clinical encounter with patients, involving people in making decisions about their health, or developing clinical guidelines, using the available evidence on preferences. The evidence on preference, in turn, can have different levels of certainty, for which the GRADE approach has developed a specific methodology[10],[11].

Systematic study of people’s values and preferences is scarce, and evidence is often not available. However, clinical experience can provide considerable information about typical values and preferences. An example is the knowledge of the strong aversion of pregnant women to the risk of major fetal anomalies when receiving a drug, even when the associated risk is considered small[2]. However, there may be evidence reporting large variability or discordance among individuals’ preferences, and a single recommendation is less likely to apply uniformly to all patients. Therefore, when there is large uncertainty or variability, it is impossible to make a strong recommendation for or against, as this would not apply to the entire target population (Example 2, regarding prostate cancer screening)[8].

C. Acceptability and feasibility

An intervention considered beneficial may not be acceptable for various reasons, whether personal, community, or religious; or it may be considered unfeasible because it is incompatible with the local reality for which it was originally intended (Examples 3a and 3b).

D. Resource use, equity, and cost-effectiveness

There are circumstances in which resource use is a key factor in the development of a recommendation, especially when the balance between benefits and harms is narrow or the implementation of the recommendation has important budgetary implications (Example 4). Similar to values and preferences, there is often little information on the economic implications of recommendations, especially considering the direct and indirect costs involved. The EtD frameworks indicate that some key factors should be considered in estimating resources for the formulation of recommendations:

a) The funder’s perspective in estimating costs.

b) The funder’s perspective on resource use.

c) Local (jurisdictional) variations in cost estimation.

d) The perspective in defining the opportunity cost (what is left out to finance the implementation of the recommendation).

e) Political implications in the allocation of resources.

There are two tools of health economics that allow this criterion to be assessed:

1. Cost-effectiveness: This domain has the purpose of evaluating what is the relative or incremental benefit of an intervention with respect to another in relation to the associated costs (incremental cost-effectiveness ratio or ICER). The ICER is a relative measure, but it does not estimate the total cost of implementing the recommendation.
2. Budgetary impact: There are interventions that are cost-effective but involve a large absolute cost, either because the disease is very prevalent or because they involve prolonged treatments, among other reasons.

Finally, when resources are used for one intervention, they are not used for others (opportunity cost), potentially increasing inequalities between individuals, affecting equity. For example, suppose there is a highly effective or moderately effective drug for a group of people within a country whose target population is small, and the drug is expensive. In that case, this intervention can displace the budget allocated to measures with a greater impact on society (vaccines and water sanitation), increasing social gaps by affecting more vulnerable groups[15].

Overlapping of the different domains

Acceptability, feasibility, and cost-effectiveness depend on efficacy. An intervention will be more acceptable if this same intervention is very effective. In turn, if the intervention is very effective, the cost-effectiveness ratio will be better. There is interdependence between the evidence and the criteria or domains to be considered, which justifies their separation within the EtD framework.

Recommendations

The recommendations formulated by the EtD frameworks can be classified according to their strength (strong or weak) and according to their direction (for or against). To summarize, the strength of a recommendation reflects the degree of confidence that the desired effects of an intervention outweigh the undesirable effects or vice versa. In strong recommendations, the balance between benefits/harm, considering also costs, and patient values and preferences, is clearly determined. In weak recommendations, the opposite is true. It is important to clarify that the term “weak recommendation” can be interpreted negatively by associating the word weak with weak evidence. In order to avoid misunderstandings, the GRADE group proposes to describe weak recommendations with other terms, such as conditional and discretionary. If any of these variations are used, authors must be consistent throughout their guidelines and the rest of the guidelines they produce[2].

It should be noted that even when confidence in the estimate of effect is low and/or the desirable and undesirable consequences are closely balanced, GRADE encourages expert panels to overcome doubts and formulate recommendations, which will inevitably be weak[2]. Table 1 summarizes the significance of these recommendations for patients, clinicians, and policymakers.

Table 1.

Summary of interpretations of EtD recommendations.

Full size

If we take the case of the strong recommendation in favor of early antibiotic use in patients with sepsis (Example 1), it is expected that most patients will receive this medication and that most (if not all) clinicians will take this course of action. Indeed, the proportion of patients with sepsis receiving antibiotics early could be a definite quality-of-care indicator for hospital policy developers.

In contrast, in the case of the weak recommendation against prostate cancer screening (Example 2), there is variability in patient values and preferences, so the decision is expected to be individualized. In this case, decision support tools (decision aids) could be incorporated to assist the dialogue with patients[17], with the use of such tools for communication being a possible quality indicator. The proportion of men screened with PSA could not be an indicator of quality because variable implementation is reasonably anticipated.

Recommendations should not be regarded as rules or rulings. Even strong recommendations based on evidence of high certainty will not apply to all circumstances or all patients. Guideline users may reasonably conclude that following particular recommendations based on high certainty of evidence will be wrong for some patients considering their unique characteristics[18].

Strength and direction of recommendations

There is no direct relationship between the certainty of evidence and the strength of recommendation. The certainty of the evidence is assessed for each outcome; usually, several outcomes are estimated for an intervention, and in addition, all the other criteria already mentioned should be considered. For example, an intervention with high certainty of evidence for its effectiveness may end up with a recommendation against it due to uncertainty about its safety (about its adverse effects). An example of this is the use of quinolones to treat urinary tract infections during pregnancy[19]. Conversely, an intervention for which there is low certainty of evidence for its effectiveness may end up with a strong recommendation in favor of, for example, the duration of handwashing[20].

Adaptation and adoption frameworks

There are many recommendations on prevention, diagnosis, and treatment of health or public health problems developed by teams that systematically apply this methodology. To avoid unnecessary efforts, the GRADE group created the adaptation–adoption frameworks, making it possible to adapt or adopt recommendations made in other countries. For example, in Argentina, the breast cancer screening recommendation (Example 4) was developed from other current recommendations that used the GRADE method using the adaptation–adoption process developed for this purpose[4] (Figure 3).

Initially, we search for relevant guidelines with GRADE methodology, and we can update the evidence that supports them. Then the EtD framework is reassessed in the local context. If the recommendation is similar, it is called “adoption”, if it is different, “adaptation”, and in the case of not having GRADE guidelines, one can choose to develop new recommendations from scratch or “de novo”.

Figure 3.

Summary of the process of adaptation/adoption of recommendations.

Full size

Conclusions

The available evidence on the effectiveness of interventions should be considered together with evidence of other types, mainly people’s values and preferences. Specific aspects of the applicability of an intervention, such as acceptability, feasibility, and costs, should also be included. As this process is developed transparently, the EtD frameworks are adoptable and adaptable by other territories.

The certainty of the evidence is not directly proportional to the strength of the recommendation, given the multiple facets of health problems.

As in the first part of our description of the GRADE method, using simple descriptions accompanied by examples, we seek to provide a simplified and summarized approach to this methodology for its understanding and application by the general public, students, and novice methodologists.