Introduction

Clinical decisions in critically ill patients involve high complexity, limited availability of resources, and risks to patient safety. In intensive care units, uncertain diagnoses and multiple comorbidities contribute to errors with severe consequences. For example, in the United States, between 40 000 and 80 000 people die each year from diagnostic errors in these units [1]. The emergence of SARS-CoV-2 intensified this uncertainty to the point that the term “ blindness COVID” was coined [2]. In Chile, supply constraints, with fewer intensive care unit beds than the average for the Organization for Economic Cooperation and Development (OECD) [3], prompted the integrated public-private management of critical beds through specific regulations [4]. In this scenario, understanding how intensivists make decisions and the factors that influence them is particularly relevant.

The literature on decision-making in intensive care units during the COVID-19 pandemic is limited, especially regarding cognitive processes [5,6]. Substantial variations have been described in the management of patients with severe COVID-19 [7], in interhospital mortality [8], and in the care of acute respiratory distress syndrome [9]. This variability calls into question the hypothesis of strictly protocolized practice and suggests the influence of contextual factors and cognitive shortcuts.

This study is framed within two complementary conceptual frameworks developed by Herbert Simon: the role of contextual factors in the decision-making environment and the architecture of cognitive processes [10]. The first involves taxonomies that distinguish between workplace, patient, and professional factors [11,12,13,14], with some authors pointing to context as the most relevant determinant of clinical reasoning [15]. As for the cognitive process, Kahneman’s Dual Process Theory distinguishes between a fast, intuitive System 1 and a slow, analytical System 2 [16,17], whose complex interaction can lead to heuristics and cognitive biases. Alternatively, other authors emphasize the adaptive role of heuristics as effective frugal strategies under limited information [18]. Using heuristics in clinical decision-making does not necessarily imply making wrong decisions [19].

Given the difficulty, for ethical and logistical reasons, of studying decision-making in actual clinical practice, clinical vignettes are a widely used method for investigating decision variation under controlled, comparable conditions [20]. Of the more than 100 heuristics described [21], this study focuses on three: statu quo/omission [22], representativeness [23], and availability [24]. These have clinical plausibility and prior evidence, having been described in the decision-making process for critically ill patients [25,26,27].

The COVID-19 pandemic created an ideal environment for context and heuristics to influence decision-making: high pressure to provide care, incomplete information, frequent interruptions, fatigue, and high emotional stress [28,29]. This environment can lead to systematic biases, such as a preference for maintaining previous behaviors (statu quo) [30,31], overweighing recent or vivid cases (availability) [24], or assessing the probability of an event occurring based on its degree of similarity to a clinical pattern (representativeness) [23,26]. There is no specific existing empirical evidence, either in Latin America or particularly in Chile, on the use of heuristics in the clinical decision-making process in intensive care units during the pandemic. This study aims to improve this situation with three specific objectives:

1. To determine the variability in the decision-making process by intensive care unit physicians during the COVID-19 pandemic in Chile.

2. Analyze the association of certain contextual variables related to the workplace, the patient, and the physician with clinical decision-making.

3. Estimate the frequency of use of status quo, representativeness, and availability heuristics in clinical decisions.

The object of study is, therefore, the clinical decision-making process of Chilean intensivists during the COVID-19 pandemic.

Methods

Study design, scope, and participants

This is a quasi-experimental cross-sectional study of a qualitative-quantitative nature using an online questionnaire adapted via the Otree platform [32] (Appendix 1). The questionnaire was distributed among Chilean intensivists affiliated with the Chilean Society of Intensive Care Medicine (Sochimi) through this association. In addition, a snowball strategy was used to recruit intensivists who met the study profile. Participating physicians received a questionnaire that included sociodemographic data and vignettes describing clinical cases and decision alternatives, accessible via a link or QR code in the email.

The study design maintains the anonymity of the person completing the questionnaire. Five clinical vignette models were designed by eight specialists, three in intensive care responsible for care during the COVID-19 pandemic, and five professors from the Faculty of Economic Sciences at the University of Granada, experts in behavioral economics and health economics.

The vignettes were developed following the recommendations for vignette development by Peabody [33], Converse [34], and Spalding [35]. The content of the vignettes is based on a review of the scientific literature, the authors' clinical experience, and other previously published vignettes [36,37].

The vignettes were validated and piloted by international intensive care specialists at La Paz Hospital (Spain) and by specialists in intensive care units at Puerto Montt Hospital (December 2021 to March 2022). The project was approved by the Ethics Committee of the University of Granada (Spain) and by the Ethics Committee of the Reloncaví Health Service of the Ministry of Health (Chile).

The vignettes included scenarios with contextual variables depending on the context (time of day when the decision was made, pandemic wave), the patient (age), or the physician (clinical information available on the patient). Each vignette presented two decision alternatives, one of which involved the use of a heuristic (statu quo, representativeness, or availability; Table 1, Appendix 2).

The questionnaire was answered by 51 physicians (specialists and residents) who treated patients in intensive care units during the COVID-19 pandemic in Chile, in both the public and private systems (June 2023 to June 2024).

All cases were presented to all physicians in random order, with a time limit established in the pilot study, after which the application presented a new vignette if there was no response, assigning a missing value for that case and physician. The response time for each case can be found in Appendix 1.

Data analysis

For the first objective (analysis of variability), absolute and relative frequencies of the possible answers are used. The cases will show greater variability the closer their answers are to 50%.

For the second and third objectives, the mid-P version of the McNemar test is used to determine whether the differences are significant in each case. We start with a contingency table of responses to the same vignette across the two possible contextual situations. The discordant ones are defined by the counts in cells b and c for each vignette (Table 2).

In cases where fewer than 25 discrepancies are recorded in the responses to two compared cases, the results are reinforced by a binomial test [38,39].

Regarding the second and third objectives, we compared physicians' responses to cases across the clinical vignette models. We used an intra-subject design, comparing subjects' responses to one case (e.g., case A) with those to another case in the same model (e.g., case G). The difference between cases within the same model lies solely in the value of a contextual variable (e.g., case A occurs at 10 a.m., and case G at 3 a.m.). This intra-subject design was evaluated using the McNemar test (mid-P version) based on 2 × 2 tables; only discordant pairs (cells b and c) are used in the contrast. In vignettes 1, 2, 4, and 5, the option linked to the heuristic was identical across versions, so the coding remained constant (0 when participants selected the option without using the heuristic, 1 when they selected it using the heuristic). In vignette 3, the option “orotracheal intubation and admission to intensive care units” was adopted as the reference category to unify the interpretation of the contrast between its variants.

To determine the effect size, the prevailing trend in the literature is followed, and the odds ratio is calculated as follows [40]:

OR = b/c

Results

First objective

There is significant variability among physicians' responses. Four vignettes showed low variability (more than 82% agreement among physicians), six showed medium variability (70 to 82%), and six showed high variability (equal to or less than 65%), which shows marked heterogeneity in clinical decisions (Table 3 and Appendix 4).

Second objective

Table 4 shows the results of the McNemar test for the five clinical vignettes and the 14 statistical contrasts across contextual variables.

The contextual factor of decision time (10 hours versus 3 hours) did not alter responses (A versus G; binomial test, p > 0.05; vignette model 1). Regarding the decision on whether to withdraw treatment after 30 days without response (vignette model 2), the patient’s age influenced both waves (I versus B and F versus M, p < 0.05), although the pandemic wave itself had no effect (I versus F and B versus M, p > 0.05).

Regarding the decision to admit patients with different profiles to intensive care units (vignette model 3), age was not a determining factor within each wave (J versus P; L versus C, p > 0.05), although decisions changed between waves for both profiles (J versus L; P versus C, p < 0.05). An odds ratio less than 1 indicates that, in the last wave, the preference for conservative management/sedation decreased, while the preference for admission to intensive care units increased. Regarding the availability of clinical information, the prior request for a computed axial tomography scan (vignette model 4) modified the choice in both waves (D versus N; H versus K, p < 0.05), while the wave alone showed no effect (D versus H; N versus K, p > 0.05). Finally, in the last wave, clinicians were more willing to consider alternative diagnoses to COVID-19 (O versus E, p < 0.05; vignette model 5).

Third objective

The use of the statu quo heuristic is studied in models 1, 2, and 4. In model 1, its use was not observed, even at 3 a.m. In model 2, an odds ratio of less than 1 suggests that physicians tend to be less likely to choose a wait-and-see approach (statu quo) when the patient is older. The results of the comparisons made for model 4 show that the statu quo heuristic is used when a CT scan has already been requested (Table 4).

The use of the representativeness heuristic (vignette model 3) has not been observed based on the patient’s age (binomial tests for vignettes J versus P and L versus C, p > 0.05).

The use of the availability heuristic is explored in vignette model 5. An odds ratio of less than 1 suggests that physicians are less likely to choose the option involving the use of the availability heuristic when moving from the first to the last wave.

Discussion

The results of this study provide important insights into the clinical decision-making process of intensive care specialists in Chile during the COVID-19 pandemic. With regard to the first objective, the study shows wide variability in decision-making across five different clinical vignette models. If these results are obtained in a study based on theoretical cases, it is likely that the variability in real conditions would be even greater, refuting the hypothesis that critical care follows established protocols. The wide variability described in the management of COVID-19 has already been mentioned [7,8,9]. In an attempt to reduce variability, multiple practice guidelines were developed during the pandemic [41]. However, some studies show that only half of patients receive the care established in protocols [9]. In these situations of high uncertainty, with incomplete information, the likelihood of cognitive biases increases [42].

The second objective analyzes the association between four contextual factors and the clinical decision. Two are contextual factors (the time of day the decision was made and the pandemic wave). Regarding the first, no statistically significant differences were observed between deciding in the morning (10 a.m.) and in the early morning (3 a.m.), although the decision of not intervening is more common in the early morning (21 versus 9). Certain studies have demonstrated a relationship between time of day and admission rate [43]. In our study, the hypothesis that the tendency to maintain the statu quo would be greater in the early morning (due to fatigue, loneliness, or difficulty accessing procedures) is not supported. One possible interpretation is the need in Chile to expedite the availability of intensive care unit beds amid bed shortages. Regarding the other contextual factor analyzed (the pandemic wave), statistically significant differences are observed in two of the four vignette models studied: model 3 on the admission to intensive care units of two patients of different ages and clinical conditions, and model 5 on a case suggestive of COVID-19 but with negative diagnostic tests. They suggest that physicians' decisions changed throughout the pandemic, probably as a result of greater knowledge of the disease, reduced severity, and greater availability of resources. It is noteworthy that participants were able to differentiate their behavior between the first and last waves of the pandemic, which is an indicator of the robustness of the vignettes.

Significant inequalities in treatment according to age have been described, with an impact even on mortality [44]. The results regarding age are heterogeneous in our study. When analyzing the decision to withdraw treatment in the absence of response at 30 days, it is observed that this is more likely to occur in older people (vignette model 2), which is consistent with other studies [45]. However, in our study, age was not a determining factor in the decision to admit a patient to intensive care units, unlike in other countries, where age was a criterion for admission [46].

The association between the contextual factor “clinical information available” (vignette model 4) and the clinical decision can be observed. In this vignette model, the response varies depending on whether or not a CT scan has been requested. If there was a prior request for this image, there would be a greater probability of maintaining the statu quo without intervention, whereas if it had not been requested, intervention would occur, specifically by requesting the CT scan, with a statistically significant difference. The study by Aberegg (on which this vignette is based) obtained similar results [37].

The third objective estimates the frequency of heuristic use in clinical decision-making. Critical care has characteristics that make it particularly susceptible to the use of heuristics: decisions must be made quickly, often with incomplete information, while caring for multiple patients simultaneously, with frequent interruptions and distractions, and under a high degree of emotional overload and fatigue [47,48]. Heuristics would be evidence of how people “satisfice” their information needs, without having to find the “perfect” answer, which is very difficult to obtain under circumstances such as those described [49,50].

The statu quo heuristic, defined as the tendency to maintain a given situation without intervening, is supported by solid evidence [51,52]. Of particular relevance is the work by Aberegg [37], which demonstrated statu quo bias in critical care. In our study, the result is mixed. Its use is not observed for the time difference (vignette model 1), but it is observed in the options of maintaining or withdrawing treatment after 30 days without improvement (vignette model 2), or when the decision is analyzed according to the clinical information available (vignette model 4).

The study found no evidence of the representativeness heuristic (vignette model 3). In this regard, age is not interpreted as a sign of poor prognosis that would lead to deprioritizing admission to the intensive care unit for an older person in good clinical condition.

Finally, the availability heuristic is observed, especially during the first wave, leading most physicians to consider any clinical presentation as SARS-CoV-2 infection, even when diagnostic results are negative (vignette model 5). This bias could result in overlooking other equally serious conditions [36].

This work has several limitations. First, the Hawthorne effect (awareness of being studied) and social desirability bias should be noted. Despite the meticulous validation process, conducting a study using vignettes may be considered a limitation to external validity, although most studies on heuristics use this methodology given the difficulty of conducting them under real-world conditions [53]. Used for more than 30 years, clinical vignette studies yield better measures of quality of care than medical record reviews when assessing differential diagnosis, test selection, and treatment decisions [54]. The simultaneous use of several heuristics in the decision for each case also cannot be ruled out. The study is not representative of all Chilean intensivists; logistical and economic constraints precluded conducting a study with a representative sample of the country’s professional population, leading to recruitment through scientific societies [55]. The case sample is small for demonstrating external validity, but it allows identification of findings that justify initiating an in-depth line of research. Although it is true that other factors could influence clinical decision-making, keeping cases constant and allowing variation only in the variables studied, in our view, favors the precision of the study’s objective measurements. The unavoidable accumulated knowledge about COVID-19 throughout the pandemic, rather than a bias, is considered a highly relevant contextual factor, confirming that decisions differed depending on that knowledge.

The results identify opportunities to improve critical care during pandemics. First, they suggest the need to update and simplify care protocols to reduce the observed variability. They also point to the formulation of strategies to improve intensivist training, particularly in recognizing cognitive biases and heuristics in high-pressure contexts, and to promote the implementation of clinical case analysis activities.

The complexity of clinical decision-making, and its consequences in terms of safety, cost, and survival, make it necessary to improve knowledge about the influence of cognitive biases in this process.

Conclusions

Chilean intensive care physicians participating in the study show wide variability in their clinical decisions during the COVID-19 pandemic. In addition, the association of certain contextual factors with the decision (pandemic wave at the time of the decision, patient age, or available clinical information) is demonstrated. Finally, the use of certain heuristics (availability, status quo) in certain circumstances during intensive care unit care during the pandemic is observed.

This study is the first of its kind conducted in Chile and one of the first internationally. Despite its limitations, its results contribute to the analysis of the “black box” of clinical decision-making in contexts of uncertainty, allowing for the formulation of strategies for improving the training of intensive care specialists. This could be useful for optimizing clinical management in future pandemics or endemic problems in Chile (e.g., hantavirus infection).

The study and its limitations allow us to propose future lines of research in this area, expanding the casuistry to a larger number of intensivists and residents, exploring other contextual factors, analyzing additional heuristics, and comparing results with those from other countries.

The need to deepen our understanding of decision-making, the influence of context, and the use of heuristics is transversal to any medical discipline.