Justification

The use of digital devices has been controversial in Catalonia and elsewhere. The use of these devices appears to have increased due to the COVID-19 pandemic, the proliferation of social media, and technological advances, including artificial intelligence. At the same time, their use has been linked to declines in school performance [1] and an increase in mental health disorders, particularly among young people, causing social alarm, in the health and education sectors, as well [2].

Several entities in Catalonia, such as the Catalan Society of Pediatrics [3] and the Department of Education and Vocational Training [4], have taken steps to address this problem. The Official College of Physicians of Barcelona (COMB) has published the document "Digital Protection for children and teenagers [5]" (hereinafter referred to as “the COMB document”) as "the first compilation of recommendations and requests for action emerging from the COMB working group and aimed at the public authorities, telecommunications operators, application developers and families" (page 5).

The great social importance of the COMB document cannot be overemphasised, although, to optimize the benefits and minimize the risks of the recommendations, these should be developed using systematic and transparent methods [6]. In our opinion, this is not the case of the COMB and we describe here the evidence supporting our assertion, based on a scrutiny of the content of the COMB document.

Methods

There are tools for the methodological evaluation of health guidelines or recommendations, such as Appraisal of Guidelines for Research & Evaluation (AGREE-II) [7]. However, it only makes sense to apply these tools to evaluate guidelines that minimally describe some methodology for their development. When there is no methodology, or no methodology has been described in the guideline to be evaluated (as is the case of the COMB document), AGREE-II loses its specificity by classifying all, or almost all, items as lacking information.

Our interest focuses on verifying the content of the references cited, which could correspond to items 7, 8, and 9 of AGREE-II: use of systematic methods for searching for evidence, description of the criteria for selecting evidence, and description of the strengths and weaknesses of the evidence, respectively. To go into detail on these aspects, it is necessary to conduct a more thorough verification of the references cited in the COMB document.

The tools for analyzing the integrity of references in biomedical publications have traditionally been based on citation counting and, more recently, on rather sophisticated tools [8]. Our content analysis consisted of comparing the statements in the COMB document with the statements in the cited sources after a thorough reading of the latter, in order to identify:

• Irrelevancies: statements in the COMB document that are found in the cited sources but are irrelevant or do not fit the main theme of this document.

• Biases due to misinterpretation: statements in the COMB document that cannot be supported by the findings in the sources cited.

• Biases due to the selection of evidence: statements in the aforementioned document that are found in the cited sources, but in which other relevant findings qualify or contradict them.

We also found errors in the bibliographic citations, which we have documented.

We describe the problematic issues in the COMB document in the order they appear in the text to facilitate their follow-up. Additionally, we also offer a summary of these issues classified by type of bias in Table 1.

Results

Firstly, we should note that the COMB document addresses fundamental issues related to the health of children and adolescents regarding the use of digital devices.

In relation to the section on “Cognitive and social development”, based on reference 3 (erroneous citation, actually referring to Madigan 2020 [9]), the document states that "young children who spend more time in front of screens are more likely to have delayed language development and fewer language skills." This statement suggests a causal relationship that is not supported by the meta-analysis cited: the 42 studies included are observational, and the meta-analyses, as emphasized by the study authors, are based only on correlations. The authors also describe beneficial effects related to "quality" screen time, which the COMB document omits. In addition, the studies included in the meta-analysis have considerable methodological limitations (for example, only 13.5% of the studies used a validated instrument to measure "screen time") and, as the authors acknowledge, there are multiple predictors of language development that must be taken into account that have been ignored in the meta-analyses. In most of these studies, exposure included "television" (only 2 of the 42 studies focus on mobile phones, which are the main digital devices today), and the meta-analyses included mixed studies published over nearly 50 years, since 1973, compromising the applicability of the findings to today’s digital environment.

Next, the COMB document mentions a study that links overexposure to screens with decreased attention span [10]. The limitations of this work are significant and reported by the authors themselves: the study is observational and retrospective, based on children who were around seven years old in 1996, 1998, or 2000; the attention scale does not necessarily have clinical validity; and screen exposure (i.e., only television, largely outside the focus of the current digital environment of childhood and adolescence) was elucidated by asking mothers in 1990.

The COMB document does not overlook an important aspect: aggressive behaviors. In this regard, it cites Ferguson’s 2011 study [11] (mistakenly attributing it to the journal Pediatrics). The COMB document misses the fact that the study sample comes from a small town in southern Texas (USA) and is therefore not representative of any particular community. A reading of the study does not seem to support the claim made in the COMB document that overexposure of minors to screens and aggressive behavior are related. In fact, the aggressive behavior in the study refers to guardians or caregivers, not minors. Instead, the study focuses on the relationship between television and video games, on the one hand, and attention problems, on the other, and finds no significant associations.

The COMB document cites a study with a non-representative sample of 69 parent-child pairs, which analyzes the correlation between screen use and brain images [12]. We do not believe that these findings constitute "sufficient evidence" (as stated in the COMB document) of a relationship between screen use and reduced opportunities for face-to-face interaction in the context of the development of social and emotional skills in childhood.

In the section on “Mental Health”, the COMB document cites a relevant reference, namely Purba et al.’s 2023 systematic review, to support the claim that "social media use is associated with increased [seven] health risk behaviors in adolescents." However, the review itself establishes the degree of certainty of the evidence, which is "very low" for six of the behaviors. This uncertainty is due to the risk of bias in many of the studies included in Purba 2023, which tends to favor the association between exposure and risk behaviours. This is a relevant detail that is overlooked in the COMB document.

The authors, title, and Digital Object Identifier (DOI) of reference 8 in the COMB document are incorrect. The DOI corresponds to a completely different study [13] and the authors are actually Primack et al [14]. This study suggests an association between social media use and social isolation (not self-esteem, as indicated in the COMB document) among adults aged 19 to 32 (not adolescents, as suggested in the text). Although the COMB document addresses the issue of cyberbullying, the statement about the "risk of cyberbullying" does not appear to be based on the study cited.

The COMB document goes on to claim that there is a "proportional" relationship between the time spent using social media and electronic devices and symptoms of anxiety and depression, citing Twenge et al [15]. In fact, this study only presents correlations focusing on depression and suicide. The study presents other quite relevant findings, such as gender differences and considerations of the participants' socioeconomic environment, which are ignored in the COMB document.

The relationship between social media and eating disorders is discussed in three paragraphs in the COMB document. The first two paragraphs are of great interest, but remain speculative. The meta-analysis [16] cited below is not, in fact, a meta-analysis but rather a thematic analysis of studies, and is used to highlight the negative effects of social media, while ignoring the fact that the review itself also reports other beneficial effects. In the following paragraph [17], a study of Norwegian adolescents is cited and reported accurately. Another study [18] is then cited to suggest a link between Instagram use and body self-esteem, omitting that the study participants were male adults, outside the focus of the COMB document. The link between social media and eating disorders is again argued based on another study [19].

The COMB document rightly addresses aspects of physical health. In this regard, the paragraph on "sedentary lifestyle, obesity, and sleep problems" cites a meta-analysis that, in fact, is an observational, cross-sectional study [20]. Furthermore, it is not related to childhood obesity, contrary to what is indicated in the COMB document, most likely due to an error in the citation. The following reference [21] on childhood obesity and eating habits is not a meta-analysis but a review of 13 systematic reviews, with narrative results of varying quality that require careful interpretation.

Regarding sleep habits, the COMB document cites a study in which the authors, title, and DOI are again inconsistent. Based on the title, the study [22] presents a wide range of sleep-related indicators. Although most of the effects are harmful, the COMB document fails to report inconclusive associations and underlying methodological limitations. The last study [23] cited provides more convincing evidence, although we have not found any statement in it about "learning," as suggested in the COMB document.

Concerning visual fatigue and myopia, the COMB document cites a study [24] (with an erroneous DOI and journal) that shows an association between near vision and the development of myopia. This is an important point that the COMB document rightly highlights, correctly stating that near vision includes the use of screens. However, the study points to a wide variety of digital and non-digital activities classified as "close work," which determines the interpretation of the findings. Furthermore, it is worth distinguishing between the different aspects that can affect visual health, such as distance (as suggested in the COMB document, although it only refers to digital devices), but also the type of brightness of these devices, the environment [25,26], and the activity performed [27,28]. The multifactorial nature of these effects [29] is ignored in the text analyzed.

Finally, the health care recommendations (page 14) are not based on any evidence about which interventions may or may not work and in which aspects of the problem. Furthermore, the recommendations in the document focus on communication or are obvious (e.g., "making the pediatrician a reference point"). The COMB document is not the only example of recommendations related to digital health that lack evidence of the effects of interventions [30].

Conclusions

It is commendable that the COMB has taken the initiative to develop recommendations for an issue of great social, educational, and health relevance. However, this document lacks the precision and reliability necessary to inspire confidence in the Catalan society and to serve as an instrument for informing policies and practices, considering the numerous errors in the bibliographic references, the unjustified selection of evidence [31], the reporting bias in considering the findings to the studies [32], and the failure to acknowledge gaps in the evidence. The clinical basis is unacceptably biased, and there is no evidence on the effects of interventions. Furthermore, the COMB document lacks a methodological section, failing to provide at least some transparency, or a statement of the authors' actual or potential conflicts of interest.

In the field of health, all recommendations must be based on the best available evidence and formulated through a systematic and transparent process [33]. This is true for biomedical interventions, and we see no reason why it should not also hold true for digital health interventions, such as those affecting the digital protection in childhood and adolescence.

In light of the above, we request that the COMB withdraw these recommendations from the public domain and produce new, evidence-based recommendations on digital health, using a logical framework that allows differentiation among types of digital devices, mechanisms of action, and expected outcomes. In addition, we suggest following a systematic and transparent method with clear questions and a critical review of all available evidence [33].