Background to a controversy

On 6 July 2010, on behalf of a broad political spectrum, a group of 10 Members of Congress of the Republic of Chile presented a bill that seeks to prohibit the production, import, marketing and distribution of vaccines containing thiomersal or organomercury compounds, regardless of their concentration. This was prompted by the possible risk of developing autism or other complications, particularly in vulnerable groups such as pregnant women and small children.

The bill was met with little opposition during first round of discussion, despite a few dissenting voices coming from the Country’s Infectology and Pediatric Societies. Nevertheless, most of the experts called up by Parliament categorically threw their support behind prohibition [1].

Over the last year, the stance arguing that scientific evidence shows no association between thiomersal and autism began to gain momentum. One of the factors that appears to have had a decisive influence on the opinion of some experts ‒especially the representatives from the Chilean Ministry of Health‒ was the recommendation from the Global Advisory Committee On Vaccine Safety, an independent committee whose mission is to advise the World Health Organization in matters pertaining to vaccine safety, and who in 2012 held a meeting with the purpose of assessing thiomersal use.

Despite the divergent stances, last 17 January the Chilean Parliament gave their go-ahead to the bill in question, on the grounds of the precautionary principle, owing to the lack of certainty with regard to the safety of thiomersal.

The passing of the bill reaped organized disapproval by the country’s scientific societies, the Ministry of Health, and a sector of civil society, with resonance even among the international scientific community [2]. The questioning directed toward the earlier process (namely a few discrepancies in the literature review used as the backbone for the legislation, but particularly a coordinated stance among the players against prohibition) led the President of the Republic to issue an additive veto. The next step will now be in the Senate of the Republic: either deciding on approval of the presidential veto ‒which will provide a new opportunity for deliberation‒ or rejection, thus confirming the decision to ban thiomersal use in vaccines to be used in Chile.

Not intending to provide a systematic review or a formal summary of existing research, this article ponders the scientific evidence gauging the association between thiomersal and autism. The main goal is to elaborate on the causes that have led to controversy, and to analyze the shortcomings in both form and content that have occurred during this decision-making process. The insights obtained from this case make it possible to outline a constructive path, both for the possible reopening of discussion on thiomersal use in Chile as well as for future decision-making instances vis-à-vis health policies.  

How do we advance toward proper decision-making from the healthcare standpoint? The importance of defining the question

The decisions made in the arena of health policies are complex. Furthermore, the instances for discussion and participation are not that many. Therefore, those taking part in the process may wish to take the opportunity to promote broader causes. Keeping the discussion focused on the question at hand is essential for constructively moving forward toward decision making.

When it comes to thiomersal, notwithstanding a series of arguments in favor and against its prohibition has been put forth, the question which has represented the focal point in the discussion is whether or not thiomersal causes autism. Unfortunately, reducing the health policy decision to this lone question constitutes an oversimplification of the problem, leaving out various fundamentals of the question at hand.

Generally speaking, it is acknowledged that a question should contain the following components:

1. What population will it affect? In this case, it is the entire vaccine-receiving population: children as part of the national immunization program, and adults during occasional vaccination campaigns (i.e., influenza).
2. What is the proposed intervention? Not only is it being proposed that thiomersal (ethylmercury) use be dropped, but also to legally ban it, thereby turning to alternative vaccines that do not use this compound.
3. What is the compared alternative? Keeping thiomersal in vaccines.
4. What are the main outcomes, benefits, risks, and costs? Albeit autism is a significant outcome, there are many others to take into consideration.
5. Where will the decision be made? Within the Chilean healthcare system.

In the last few weeks, we have seen how the discussion has deviated in different directions with regard to the original question: What is the impact of banning thiomersal by law in every single vaccine in Chile? In order to illustrate:

• When discussing the risks associated with other more toxic forms of mercury than thiomersal (such as methylmercury). This point was duly made by experts on the matter, yet it is not clear to what extent it has been established among Members of Parliament and the public at large.
• By focusing exclusively on the effects of vaccines containing thiomersal, we forget that part of the problem in this case is also what the enforced mechanism (banning) implies, as well as the benefits (or lack thereof) of alternative vaccines.
• By falling under the vaccination/anti-vaccination paradigm, by defending what vaccination has accomplished over the last two centuries, by bringing to light the latest outbreaks in countries where the anti-vaccination movement has taken a more firm hold, or by quoting the results on the debunked Andrew Wakefield study [3]. Arguments such as these are useful for making a pause in the review (for example, when explaining how the fear arose that vaccines caused autism), yet they turn attention away from the discussion.

In no way do we seek to assert that discussing these questions is irrelevant. Furthermore, it could be desirable to put forward new policies with a broader scope. However, it is necessary to concentrate first on the main question.

Global evidence, local decisions

There tends to be confusion regarding what role to assign to the decisions made in other countries in response to local issues. In order to answer this, it is essential to separate the “evidence” from the other aspects that are required in making decisions. As shall be repeated throughout this article, evidence is necessary but not sufficient in its own right for decision-making.

By “evidence”, we are referring to the results stemming from scientific research that have evaluated the effects of vaccines containing thiomersal on health (both favorable and unfavorable). Granted the transferability of the results obtained from studies among countries was a topic of debate toward the end of the last century, nowadays there is broad consensus in that the effects are consistent throughout different countries, ethnic groups, and so forth. In this case, if there was a link between thiomersal and autism, this would be observable in every country where it was assessed.

What has substantially varied throughout different places is the context of the decisions. Apart from the evidence, decision-making in health policies requires taking a series of other aspects into consideration: the relevance of the problem, the values and preferences of those who will be at the receiving-end of the measures taken, the costs, and the impact on equity, its applicability, and the feasibility of its implementation. It is this very complexity that places risk on extrapolating health policy decisions made by other players, and in other times and places.

This notwithstanding, in the case of Chile, one of the main arguments that have been used to justify banning thiomersal in vaccines was the fact that it had been banned in the United States in 1999 [4]. Apart from the many differences in the context, the data we currently have at our disposal is totally different from what existed back then, as can be seen in Figure 1 [19].

Figure 1.

Evidence matrix regarding the link between thiomersal and autism

Full size

The columns represent epidemiological studies on humans, and the rows represent existing systematic reviews [11], [12], including the report from the Global Advisory Committee On Vaccine Safety [13]. The green boxes represent the studies which were included in each review. The light blue row applies to the review from which the evidence matrix was created at Epistemonikos.org.

The matrix shows that the existing body of evidence nowadays is massive, as well as being greater than what any of the players has presented. On the other hand, there was no evidence coming from any epidemiological studies in 1999, which was the year when it was banned in the United States. Neither is there any systematic review encompassing the entire corpus of existing evidence.

In sum, the scientific evidence regarding the effects ‒or in other words, the risks and benefits of the decision under evaluation‒ is indeed transferable. The decisions made in another context are not, however, insofar as they involve a series of other specific and unique factors.

Who should make these decisions?

In Chile, it is rather uncommon for a decision on the risks and benefits of a therapeutic intervention to be made in Parliament. This is especially true when it is of a predominantly technical nature, and focused almost exclusively on the merits of scientific research on a concrete fact, which in this case is the causal link between thiomersal and autism.

In the vast majority of countries, regulations are put in place preventing Parliament from discussing a matter of a technical nature. In point of fact, in Chile there are similar regulations in other areas. When this does not occur, the array of options that are weighed in discussions is reduced tremendously. In the case of Chile, if the discussion had been left in the hands of the agencies that usually make these kinds of decisions (such as the Ministry of Health), the spectrum might also have included an intermediate alternative, such as replacing vaccines containing thiomersal without implying a ban on its use. This leaves the possibility open for reconsidering the decision when further data is available, or also in emergency scenarios when the availability of vaccines containing thiomersal is insufficient and when the benefits outweigh the potential risks, if they do exist. This path would also have made it possible to request the necessary reviews and studies for making this complex decision.

The context of a health policy decision is largely determined by those who are part of this process. Among these we can find the decision-makers (who make the final decision), the experts (who act as advisors on technical aspects), other players (interested and actively participating parties), as well as the stakeholders with potential financial conflicts of interest. As far as the case on banning thiomersal in Chilean vaccines, the players are as follows:

1. Healthcare decision-maker: this lies in the hands of the Legislative Branch (Parliament), that votes on passing a bill, and on the Executive Branch (President of the Republic), who can either sign or veto the bill. In this anomalous healthcare decision-making circle, the Ministry of Health and other agencies that normally act as decision-makers have taken-on the role of thematic experts, resuming their role as decision-makers at the very end of the process, by asserting their influence on the presidential decision.
2. Experts: albeit the term “expert” has lapsed into disuse in the literature on informed evidence-based decision-making, we have retained it on the grounds that it deals with a widely-used concept within the framework of this discussion. The Legislative Branch has been seen in this matter dealing with contrasting opinions from various expert entities. On the one hand, scientific societies and the Ministry of Health, and the Chilean Medical College on the other, acting through its president, speaking as an individual or representing the institution.
   At present there are two classes of experts: the expert on the matter (the technical pacesetter on the issue being discussed, which in this case applies to infectologists, immunologists, pediatricians, neurologists, and so forth), and the methodological expert (the expert in compiling and summarizing the necessary evidence, with consideration toward the remaining involved factors as well the way in which the decision-making process is conducted).
   As far as we know with regard to the Chilean case, no methodological expert took part in this process.
3. Other players: anyone with a vested interest in the decision (stakeholder), and who may have actively taken part in this process.
4. Stakeholders with financial conflicts of interest: anyone who stands to be financially affected by the decision ‒be it in favor or against. The most evident conflict of interest is that of the laboratories who would benefit from banning thiomersal in Chilean vaccines; however, there could be many others. In Chile, there is no proper regulatory framework mandating a declaration of conflicts of interest within the context of healthcare decision-making, nor is there a lobbying law barring laboratories from influencing members of parliament.

Table I shows how different players have behaved with regard to the case in Chile. It is worth pointing out that, as far as we know, there are no players that have stated having an intermediate position. This suggests that either recourse ‒be it keeping or banning thiomersal‒ is a reasonable position.

Table I

;;;;Table I. Main players involved in the decision to ban the use of thiomersal in vaccines along with their stance on the issue.

Full size

The evidence is necessary but not sufficient for making healthcare policy decisions

Proper analysis of the evidence is governed by rigorous methods requiring specific methodological expertise. In Chile, there are multiple academic institutions that can provide such expertise and that actively collaborate with the Ministry of Health and its agencies with a view to ongoing progress in improving decision-making processes. As an example, the Ministry is putting the finishing touches to a methodological manual that uses the Grading of Recommendations Assessment, Development and Evaluation methodology (GRADE for short; the same one we used for this analysis), in order to steer the development of future practical clinical guidelines. In addition, the very Ministry of Health has promoted having a health technology assessment agency which could provide technical data for informing on these kinds of decisions. Albeit there is further to go before this, shifting these kinds of decisions to Parliament entails a major step backward in terms of collaboration between government and academia.

Just as we have reviewed, the scientific evidence regarding the risks and benefits of action is but a mere piece in the decision-making process. Focusing the decision exclusively on this leads to a toughening on stances and a selective or subjective use of evidence for justifying the standpoint which is being defended. The explicit acknowledgement of this fact would have made a major difference in the Chilean case, especially if a structured decision-making model had been used, such as the GRADE model [5]. This model has been used by two of the most often cited organizations during the Chilean discussion: The Advisory Committee on Immunization Practices of the Centers for Disease Control and Prevention [6], and the World Health Organization [7]. In short, this model aims to clearly define the problem at hand, to include the values and preferences of the patients that will be affected by the decision, to analyze the magnitude and degree of certainty of the risks and benefits, to assess the necessary resources and the incremental cost, to take the impact on equity into account, and to consider how acceptable and feasible is the course of action.

Why so much controversy on the risks of thiomersal?

In the case of thiomersal, the “evidence” appears to reveal apparently opposite results, depending on the stance of the person quoted. Without going into detail, we will take a look at some of the explanations for this phenomenon, as well as how it has compromised the information received in Parliament.

The first fact that comes to attention is that there is no clear list on what evidence has been analyzed, let alone a proper summarized body of the abundant existing data. Nor is it clear whether all the experts have enjoyed access to the same data.

On the other hand, some experts have cited studies that others have rejected as fraudulent. The most representative of these cases are the studies by Dr. Mark Geier [8], a controversial American doctor whose studies have been cited by some of the experts who spoke before the health commission, among them the president of the Chilean Medical College [1].

How does one gain access to “the evidence”?

Though it may seem outlandish to the public, nowadays an enormous amount of effort is required to be able to identify and summarize everything we know with regard to any given health intervention. The amount of data has soared dramatically. Some data is unavailable. There are fraudulent and poor-quality studies, and the methods for performing a combination and summarization of all the studies are ever more sophisticated.

The type of summarization which is used at international level to sum-up what we know regarding a specific health question is called systematic review. In other words, it is data that has been prepared using a systematic approximation which is documented in the Methods section, in order to avoid bias and error [9]. Systematic review is increasingly being used for decision-making in health policy, particularly to support outlining the different recourses and describe their impact. To this end, systematic reviews offer four key advantages over individual studies [10]:

1. Since they are more systematic and transparent, they reduce the probability of the decision-makers being misinformed by the research.
2. By increasing the amount of identified studies, there is increased trust among decision-makers with regard to what can be expected.
3. Without the need to go over a large amount of studies, it allows decision-makers to include the applicability of systematic review findings to the specific context of their decision, as well as the feasibility of the intervention, prevailing values, and the acceptability of the potential recourses in terms of budgetary viability and political acceptance.
4. It allows all players, including public and civil society stakeholders, to constructively appraise the evidence stemming from research, given that the evidence in these reviews is structured in a more systematic and transparent manner.

The scarce use of this type of review also appears to be an important factor in the case of thiomersal. A quick search of the literature makes it possible to easily identify two of them. The first reviews the existing body of evidence up to the year 2004, which despite using systematic methods is outdated, thereby reducing how much trust we can place on its conclusions [11]. The second updates the body of evidence up to 2008 [12], yet neither does this provide the degree of certainty that a decision of this kind would require.

Figure 1 provides a comparison between these two systematic reviews, while also incorporating the report used by the World Health Organization for suggesting that thiomersal be kept in vaccines [13]. It is worth pointing out that none of the existing reviews appear to be comprehensive and that there are recent studies that are not included in any of these reviews [14], [15],[16].

Without a doubt, the simplest way of arriving at the best possible estimation on the level of certainty of the evidence would be to conduct a new systematic review.

Can we be “completely sure” of the risks and benefits of a course of action?

As we gain more and improved scientific research, we gain more certainty on what we know about the effects on health interventions. Nevertheless, absolute certainty on the risks and benefits of preventive or therapeutic interventions can never be attained. The scientific community usually reaches a consensus right about the time that “sufficient” certainty or “tolerable” uncertainty is attained, allowing something to be considered as true and build on that knowledge.  

With this in mind, the statement that there is a degree of uncertainty regarding the safety of this intervention (overwhelming for its detractors, negligible and insignificant for its supporters) should come as no surprise to those who have taken part in this discussion. Nor should it be surprising that there is uncertainty regarding the benefit that thiomersal can contribute (potentially enormous for its supporters and less significant or even ignored by its detractors).

Uncertainty is an inherent characteristic in healthcare decision-making. This was particularly well-portrayed in the words of one Member of Parliament who presented the bill and who said to the media that “no one can prove there is no collateral damage” [17]. This statement indicates a total lack of familiarity with healthcare decision-making.

To establish to what extent we can confide in the existing evidence, or what is the “degree of certainty of the evidence”, it is necessary to weigh the following:

1. The type of studies that we are using, in terms of their position within the “evidence hierarchy”. There are certain designs that under ideal conditions have a higher probability of being true, thereby having a “low risk of bias”. One example of these is the randomized control studies which are considered to be more reliable than other experimental studies. In turn, these designs are more reliable than the observational variety, and are preferred over basic sciences studies in animals or in laboratory conditions.
   Regarding thiomersal, there are no randomized control studies, nor are there any other experimental studies. All of the evidence comes from observational studies, and hence there is significant uncertainty given by the design from which the evidence is obtained. 
2. Correctly weighing the studies when we have a body of evidence from different hierarchical points.  For example, even when they are not the best, observational studies assessing the risks of thiomersal in humans hold a higher hierarchical position than studies performed on animals. Hence, when faced with the former studies, the latter cease to constitute relevant data [18].
3. Another important concept is that simple vote counting of studies in favor and against an intervention does not constitute an acceptable method. Ten poor quality studies do not necessarily weigh more than one good quality study. At present, there are far more precise summarization techniques, and that are used in scientific research on a routine basis. The method for summarizing two or more studies is called a metanalysis, and it currently represents an entire branch in statistics, especially in health. 

Final thoughts

On who makes decisions and how they are made
Albeit Chilean legislation is flexible with regard to the incumbent for making decisions on health policy, it would be desirable that the technical decisions be made at levels with an established capability and networks. In Chile, the bulk of efforts to this end have been centralized by the Ministry of Health, making it reasonable that these remain under its purview for the time being. Establishing an independent health technology assessment agency would signify an important step forward. 

It is essential to have independent methodological experts on hand for compiling and summarizing scientific evidence, and who would ideally aid the decision-making process.

On the scientific evidence
In cases where the evidence constitutes the main element driving health policy decision-making, it is necessary to have an up-to-date summary of the scientific evidence, conducted in accordance with the prevailing systematic review and metanalysis methods. These must be made public knowledge and be a topic of discussion for all players.

Albeit they may collaborate, the players taking part in the decision-making process are not called-on to compile the scientific evidence, let alone summarize or present it. In no way does this mean that their participation is not essential in all the other aspects of incidence on decisions.

On the health decisions
Health policy decisions are seldom made without a degree of uncertainty. The very concept of the precautionary principle is methodologically unsound, since all decisions have pros and cons, including the decision to take no action. In this case, either banning or allowing thiomersal use entails positive and harmful effects that must be weighed in conjunction with the remaining necessary considerations for making health decisions. 

The evidence is necessary but not sufficient for making health decisions. The considerations regarding the values and preferences of those who will be on the receiving end of the intervention, the costs (total and incremental), the impact on equity, the applicability and feasibility of implementation are all essential for making good decisions.

Notes

Potential conflicts of interest
The author has completed the ICMJE Form for Disclosure of Potential Conflicts of Interest and states that he has no actual or potential conflict of interest in relation to this article.