Introduction

Since 2020, several studies in the biomedical literature have reported a significant number of patients with persistent COVID-19. In August 2020, the World Health Organization recognized the existence of long-term effects of COVID-19 infection. It introduced the term post-COVID-19 condition to describe the range of symptoms that persist beyond the acute infection [1].

The importance of keeping track of these patients and evaluating the efficacy and effectiveness of therapeutic approaches underscores the need for instruments to quantify patients' impairment and, particularly, the impact on their quality of life. During the first half of 2020, a report was published on what seemed to be the first attempt to have an instrument to assess the severity of persistent COVID-19 symptoms in the context of rehabilitation [2]. The scale was called the COVID-19 Yorkshire Rehab Screen (C19-YRS), consisting of 19 closed and two open-ended questions. This instrument was simplified to a digital self-administered version with 15 questions, each with yes or no response options and an ordinal severity rating scale from 0 to 10 [3]. The internal consistency and some aspects of the construct validity of this instrument were subsequently reported [4].

In April 2021, the development and validation of the Long Coronavirus Disease (COVID) Symptom and Impact Tools was released, aiming to monitor the symptoms and impact of COVID-19 in patients with long COVID, understood as a distinct entity from the post-COVID sequelae of hospitalized patients [5]. These tools were developed from experiences with 492 patients from the ComPaRe cohort [6] and validated in 1022 participants from the same cohort who reported having COVID-19 at some point and had at least one COVID-19-related symptom when they entered the validation sample.

The post-acute (long) COVID-19 quality of life (PAC-19QoL) instrument [7] was published in September 2021. It contains four domains (psychological, physical, social, and occupational) and more than ten subdomains, covering 44 items in total.

Unlike other instruments that assess the quality of life or mental health of people with COVID-19, the Long Coronavirus Disease (COVID) Symptom and Impact Tools (ST and IT) were validated and have a peer-reviewed publication reporting the development and validation process. Validation findings include acceptable construct validity, where hypotheses about correlations with measures of quality of life, functional status, and perceived health were corroborated; unidimensional structure for the IT confirmed by factor analysis; high internal consistency for the IT and high test-retest reliability for both instruments. A value of 30 points on the IT is also reported as the threshold for acceptable symptomatology status [5].

The present article aims to report the translation and cultural adaptation of the Long Coronavirus Disease (COVID) Symptoms and Impact Tools for the Chilean population. In addition to this objective, the content validity of the adapted instrument is assessed.

Methods

Instruments

The Long COVID Symptom and Impact Tools, aimed at monitoring the symptoms and impact of persistent COVID-19, were developed in France based on open-ended questions for patients with persistent COVID-19 [5]. Validation was done in 1022 patients with suspected or confirmed COVID-19 and persistent symptoms for over three weeks. The Long Covid Symptom tool asks about the symptoms experienced within the last 30 days through a list of 53 symptoms grouped into clusters. The score of this tool represents the number of symptoms marked as present. The Long Covid Impact Tool contains questions about the impact of the illness in the last 30 days on six aspects of life: daily activities, family life, professional life, social life, mental state, and relationships with caregivers or health care providers. Each question is answered on a scale of 0 to 10, from no impact to major limitation of activities. The score of this instrument ranges from 0 to 60 points, and they set a cut-off point at 30 (28 to 33) to identify patients who consider their symptomatic state as acceptable.

The instrument adapted and proposed for validation is copyrighted by Tran VT and Ravaud P. The University of Chile obtained permission from Mapi Research Trust, the official distributor of the instrument, to use its English version on April 29, 2022.

Design and procedures

The cultural adaptation of the instrument was carried out in five steps, according to the best-known guidelines or recommendations [8,9]: 1) translation from the original language to Spanish, 2) synthesis, 3) back-translation from Spanish to the original language, 4) review by the editorial board, and 5) testing of the adapted questionnaire.

Three bilingual translators, with Chilean Spanish as their native language and experts in the field, translated independently and in parallel. Written instructions and the document were prepared and given to each translator. The research team reviewed the three translations, and a final Spanish version was produced. The final version was handed over to the translators for final approval before the next step.

The back-translation involved three bilingual translators who were experts in the field, one of whom was a native English speaker. An instruction manual was prepared, and they were spared from having the original English instrument at hand. Two research team members reviewed the results of the three back-translations together, looking for similarities and differences between the two versions and assessing whether discrepancies might reflect problems in the understanding of the questionnaire. Three questionable items were discussed with a native English speaker.

A committee of experts (step 4) reviewed the Spanish version after back-translation to ensure that the language used in the instrument was appropriate, understandable, and culturally adapted to the target population. This committee reviewed the semantic, idiomatic, lived experience, and conceptual equivalence of the translated COVID-19 IT instrument, made necessary modifications for inappropriate or unclear items, and reviewed the clarity of the instructions. The committee consisted of three clinical experts, a journalist with a degree in literature, and a technical advisor in charge of community epidemiology, particularly COVID-19 rehabilitation. The modifications proposed by the committee were incorporated into the translated version and endorsed by the committee itself.

The committee also reviewed the translation of the symptom questionnaire (ST) and proposed modifying the initial translation of the symptom Hypoesthesia (decreased sensitivity to touch), where the words ‘to touch’ had been omitted. The translation of the ST instrument was terminated at this stage as it was considered inappropriate to test it with ten sick subjects in the following test.

Step 5 (testing of the adapted questionnaire) consisted of testing the new version of the IT instrument on ten subjects after signing an informed consent form. These subjects came from the community and are users of three Primary Health Care Centres in the commune of Pedro Aguirre Cerda. This commune was chosen because the research team had previously conducted a prospective cohort study on anti-SARS-CoV-2 neutralizing antibody response [10,11]. They were intended to be subjects with normal cognitive abilities, with a history of having had COVID-19 within a maximum period of three weeks and still having symptoms that could be related to COVID-19. Two authors (PG and AS) interviewed the subjects after completing the ST and IT questionnaires. In this interview, they were asked about their understanding of the questions and the language used in the IT instrument, using a rubric with seven questions jointly developed by the authors. Each question was evaluated on a scale of 1 to 4, where one represented the most adverse opinion and four the most favorable (Table 1).

In addition to these seven questions, they were asked two more questions: 1) what other areas do you think should be included in the instrument, and 2) on a scale of 1 to 5 (where one is very uncomfortable and five is very comfortable), how comfortable do you feel answering the questions in this instrument?

Once this phase was completed, the authors evaluated the results of the interviews and the questionnaire responses.

Content validity

For content validity, we identified 14 experts who agreed to participate. These experts had to be doctors, nurses, or other healthcare professionals working directly with patients diagnosed with COVID-19 and persistent symptoms (long Covid), as defined by the WHO. They also had to have more than one year of experience in the care of these patients. Each expert was given a written explanation of the form and content of the instrument and was asked for his or her opinion on the relevance of each item and the relevance and representativeness of the instrument as a whole. In both cases, the experts were asked to respond on a four-point scale: 1. Not relevant or representative; 2. Not very relevant or representative; 3. Relevant or representative; 4. Very relevant or representative). To assess content validity, the precepts outlined in the COSMIN study on definitions and measurement properties of patient-reported health instruments were followed [12,13].

Statistical analysis

The analysis of the results was based on the description of the responses of the ten test subjects (patients) who participated in step five and the 14 experts who participated in the content validation. To summarize the latter, the content validity coefficient was calculated for each item and overall [14], the Lawshe index [15] and Aiken’s V for each item [16], the latter with the confidence interval proposed by Penfield and Giacobbi [17]. This procedure was also applied to the participant´s response to the six questions on the comprehensibility of each item. Since each question was answered on a Likert scale of one to four points from the worst to the best result, it was considered that the content validity scores could also be summary indicators of the participant’s opinion on the degree of comprehension, comprehensibility, and reach of the IT instrument.

Ethical authorization

This project was approved by the Scientific Ethical Committee of the Servicio de Salud Metropolitano Sur, Memorandum N° 0343/2023 dated 04th April 2023. In accordance with Chilean law N° 20.120, the study was authorized by the directors of the primary care centers and by the relevant municipal authorities. For this first phase, written informed consent was obtained from all participants before administering the questionnaires.

The social value of this study lies in the importance of having a validated instrument for the Rehabilitation Plan for COVID-19 in Chile [18].

Results

The main outcome of this study is the two final questionnaires, which were obtained after the whole process of cultural adaptation and are shown in the Annex.

Table 2 summarizes the responses to the adapted instruments of the ten patients who undertook step five.

Five of the ten subjects reported less than 28 symptoms, corresponding to 53% of the total symptoms mentioned in the instrument. In addition to the symptoms included in the ST, seven patients mentioned additional symptoms: thirst or hunger (four patients), weight gain (four patients), and chest pain with heartburn (four patients).

The mean and median indicate a low overall impact (30% of the maximum score) on the different aspects of the IT questionnaire (Table 2). The aspects with the lowest impact were social life and family life. The aspects with the highest impact were mental state and professional life.

Table 3 shows the result of applying the rubric interview on the questionnaire with the questions displayed in Table 3 for the ten participants. Seven of the 10 test subjects responded with a score of 4, which means the best concept or approval for each criterion. Two subjects expressed one point below for question 8 and question 9, respectively. The worst scoring was given by subject 6, who gave a low score (2 points) for questions 3 (Were there any words or terms you do not know?), 4 (Do you think the questions accurately reflect the impact of the disease on your personal life?) and 7 (Do you suggest any changes to make the instrument more user-friendly or more comprehensive?). These results are presented in Table 4, which shows the values of the scores used to summarize the opinions of the test patients.

The content validity index (CVI) shows the proportion of raters, in this case, test subjects, who rated each item as 3 or 4, i.e., the proportion of subjects who consider that item acceptable at least. On the other hand, Lawshe’s index (CVR) tempers this proportion and assesses the proportion of raters above 0.5 who rated the item as adequate within that 50%. Lawshe assumes that if an item is adequate, at least 50% (0.5) of the raters must consider it adequate, so if this proportion is less than 0.5, the CVR will give a negative value. Aiken’s V considers how far the average scores deviate from the minimum value that can be given to each item.

Judging from the content validity index, Lawshe’s index, and Aiken’s V (Table 4), the adapted questionnaire (IT) characteristics in terms of comprehension, timeliness, and language are very acceptable. The ten initial test subjects responded with a rating of five to the final interview question and did not propose any other areas for inclusion in the instrument.

Content validity

Fourteen experts assessed content validity: six kinesiologists, two speech-language pathologists, and six physicians. The time of experience caring for patients with prolonged COVID ranged from 20 to 48 months (36.1 ± 8.2; IQR 4.5). One expert worked in the National Service for the Elderly (SENAMA, Chile), five in primary care services, five in public hospitals, one in a university, one in a rehabilitation center, and one in home-based rehabilitation care.

Four of the 14 experts gave a score of four to all six questions of the TI. One expert did not give any of the questions four points and gave them all either two or three points. One expert gave one point (not relevant or representative) to question 6. This question had the lowest average number of points given by the experts. The values of the content validity indices are shown in Table 5.

93% of the experts considered the overall content of the instrument to be relevant or very relevant, and 86% considered it representative or very representative, as indicated by the CVI shown in the table. Aiken’s V and Lawshe’s coefficients indicate the overall acceptable content validity of the instrument. As for the individual items, the CVI showed acceptable values for all items except item 6, and both the Lawshe index and the Aiken V showed relatively low values for items 4 and 6.

Discussion

After going through the five steps of cultural adaptation, we reached a version of both questionnaires (ST and IT) that was accepted by the test subjects, Chilean patients with a diagnosis of prolonged COVID, confirmed by their treating physicians, as well their responses to the validated questionnaires, and by this study. The expert committee also accepted the translation of the symptom questionnaire. The results were unfavorable regarding content validity, considering that Lawshe’s index and Aiken’s V gave low values for question 6 (impact regarding the relationship with caregivers or health care providers). Nevertheless, the questions on the relevance and representativeness of the instrument, in general, showed good content validity with the three indexes evaluated.

The cultural adaptation of “questionnaires” that attempt to measure general or specific aspects of health is now an important part of clinical-epidemiological and general health research. Streiner et al. [19] point out that the development of instruments that measure abstract and subjective aspects of health has become more complex since awareness of the importance of measuring the impact of treatments on people’s quality of life has increased and that the efforts of many medical disciplines are now directed at improving the quality rather than the quantity of patients' lives. Likewise, these authors emphasize that, before beginning the arduous task of developing a new instrument to measure these subjective and abstract aspects, researchers should exhaustively search existing scales for the same or similar purposes. This indication derives the need for cultural adaptation since most scales will not have been developed in the environment where they are now needed, and the perception and expression of health are related to the culture in which people live [20].

The scales that we have culturally adapted are also available in French, Dutch, and two English versions for the United States, but we have not found articles reporting cultural adaptations of these versions.

The COVID-19 Yorkshire Rehab Screen (C19-YRS) has been adapted to different locations in different languages [21,22,23]. The Italian version [21] was used without cultural adaptation; the Thai version [22] followed similar steps as in this study for adaptation, which in our study added content validity demonstrated by the content validity index; of the Chinese version, judging from the summary, they follow the process outlined by Beaton et al. [9] through the friendly guide proposed by Sousa [24].

The Post-COVID-19 Functional Status Scale (PCFS), adapted to the Chilean population [25], went through similar stages to this one in its adaptation stage (translation, back-translation, and semantic and conceptual evaluation by 29 participants who had suffered COVID-19). The authors of the article call what is assessed in these 29 subjects “face validity”; we have preferred not to use that term for the pilot test with the initial 10 patients although, according to Nevo [26], the assessment of a scale by the subjects who are the object of the scale should be considered as “face validity”. Moreover, their results were similar to ours in terms of comprehension and acceptance of the adapted scale by the test patients. Unlike us, these authors submitted the translated questionnaire to an expert committee after the pilot test and not before. The committee also endorsed the already adapted and, to some extent, validated questionnaire. However, one should not be surprised by these discrepancies in the form and order of the steps for cultural adaptations. In 2015, Epstein et al. published a review of guidelines for cultural adaptation of questionnaires [27], where they identified 31 different methods for performing these adaptations, concluding that there is no consensus on the best way to do it.

This scale has also been adapted to other countries and languages. At least one recent adaptation was performed for Brazil [28]. Here, the steps of the adaptation process were very similar to ours, only that these authors included 33 patients and 57 “expert” health professionals in the pilot test, with the same questionnaire on language comprehension and perception. The CVI for the patients in this study was 0.75 for comprehension and 0.84 for language perception, somewhat lower than that obtained in our study in similar aspects.

The Post-acute (long) COVID-19 quality of life (PAC-19QoL) instrument [7] has been culturally adapted and validated in Slovakia [29] and Spain [30]. The authors of the Slovak version refer to having followed the standardized guide for the translation process recommended by the Patient-Reported Outcome (PRO) Consortium for patient-reported instruments [31]. This guide broadly coincides with the process followed in our study and proposed by Guillemin et al. [8] in 1993 and later refined by Beaton et al. [9]. After the corresponding translations, the adapted instrument was evaluated by five patients. They do not report the results of this first evaluation with patients. However, they pass to the validation stage, where they apply the instrument to 43 patients with post-COVID syndrome and to another 43 people they consider controls; the whole study they consider as a pilot. The translation and cultural adaptation process followed the same steps Beaton proposed in the Spanish version. A review was carried out by a committee of experts who evaluated the results using a qualitative method and a pilot test with 22 patients; they did not provide quantitative results of the latter test before moving on to a validation phase.

What has been said so far about the process of cultural adaptation and its results confirms what has already been mentioned about the existing variety in the literature on these procedures. It also confirms that the steps originally proposed by Guillemin and refined by Beaton et al. are still valid. The timeliness of the process followed in our study is also confirmed.

We have some reflections regarding content validity, which was proposed to assess the first aspect of the instrument’s validity. The COSMIN study (Consensus-based Standards for the selection of health Measurement Instruments), after a qualitative evaluation using the Delphi method, considers including it among the types of validity that should be evaluated in an instrument for measuring patient-reported outcomes (PRO) [13], although not all studies and guidelines include it. The definition of content validity is also subject to various discrepancies. After an extensive literature review, Juárez-Hernández and Tobón [32] found 18 definitions for this type of validity. The definition adopted by the COSMIN study seems to us to be adequate and straightforward: the degree to which the content of a PRO instrument is an adequate reflection of the construct to be measured. There seems, however, to be agreement on how to measure it in quantitative terms: the content validity coefficient, Lawshe’s index, and Aiken’s V, which are applied in this study, are the most commonly used indicators [32,33].

There is also a lack of consensus in the literature regarding the number of experts required for content validity [33]. A minimum of three has been suggested, but some authors point out it may be more than 20 [34]. Given these suggestions, our number of 14 experts seems more than adequate. In terms of requirements, experts with relevant training, experience, and qualifications in the instrument’s content are needed. Our experts were healthcare professionals with more than one year of experience in caring for patients with COVID-19 and prolonged COVID, thus, in our opinion, meeting the stated characteristics.

The content validity of the IT instrument adapted in this study showed acceptable values for the three indices mentioned above, as reported in the literature [17,35,36]. However, Lawshe’s index and Aiken’s V were low for item 6 (impact on the relationship with caregivers or health care providers). This is because five of the fourteen experts (36%) evaluated this item as little or not at all relevant to the concept that the instrument is intended to measure.

Among the scales mentioned above, created to assess the impact of post-COVID syndrome in some way, the Thai version of the C19-YRS shows a CVI of 0.95 but does not specify whether this is for all items or for the instrument as a whole. The remainder does not assess content validity.

Our study would be the first to culturally adapt an instrument that measures the impact of post-COVID or persistent COVID-19 syndrome on patients' lives in the Chilean context. The Post-COVID-19 Functional Status Scale, adapted to Chile, assesses people’s ability to perform daily living activities [37]. Both instruments can be complemented to assess the health status of patients suffering from prolonged COVID-19 in the Chilean population and its important consequences.

Among the limitations of our study is the difficulty in comparing our results with those of similar instruments since the other language versions of the instruments adapted here are unavailable. Despite known attempts, another limitation derives from the heterogeneity of ways to make cultural adaptations. Finally, the decrease in community screening makes it more difficult to detect people who may have symptoms attributable to persistent COVID-19 but lack the etiological diagnosis, thus making it difficult to recruit suitable subjects to carry out the validation stages.

Conclusions

This study provides Chilean health authorities and healthcare providers with a valuable tool to evaluate the impact of persistent COVID-19 symptoms on central aspects of people’s lives. It recognizes the significant health problem of maintaining symptoms of COVID-19 for many months after the acute phase of the disease has ended. The instrument is ready for use, although it has yet to undergo further validation.