Introduction

For the United Nations (UN), the fulfillment of the 17 Sustainable Development Goals (SDGs) in health is a great measure for humanity [1,2]. Health is included in the 13 goals that address major global health issues [3]. The World Health Organization (WHO) estimates that approximately 295 000 women die each year due to complications related to pregnancy and childbirth. In addition, there are about 2.6 million stillbirths per year and 2.7 million deaths in the first month of life. These figures highlight the importance of addressing and preventing complications of maternal-fetal care. This is a critical aspect of medical care, as it involves managing the health and well-being of the pregnant mother and the developing fetus. Clinical practice guidelines are crucial in addressing this challenge, providing evidence-based strategies to optimize resource allocation, improve healthcare infrastructure, and promote efficient service delivery [3,4,5,6].

These evidence products are knowledge translation tools that link research to action and provide conclusions that support decision-making through systematic and transparent approaches. The rigor for the development of these guidelines contemplates several methodologies, including the one proposed by the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) Working Group, which includes the analysis of the quality of evidence, the balance between desirable and undesirable consequences, the cost-effectiveness of the intervention, patient values and preferences, acceptability, and feasibility. It also addresses interventions' cost and equity impact and recommends best practices to reduce health inequities by improving accessibility for vulnerable populations. The GRADE guidelines provide a clear and consistent approach to developing guidelines, ensuring transparency in elaboration and conclusions. They stand out for assessing the relevance of outcomes of interest to clinicians and patients, considering factors such as values and preferences in their formulation. In addition, they clearly differentiate between the evidence’s quality and the recommendation’s strength [7,8]. The Pan American Health Organization (PAHO) developed the international GRADE guidelines database, which comprehensively identifies these guidelines globally, including about 80 on maternal health [9].

AGREE-Rex (Appraisal of Guidelines REsearch and Evaluation-Recommendations EXcellence) is a recent tool designed to assess the quality of recommendations contained in clinical practice guidelines. Researchers mainly use this instrument to evaluate the clarity, consistency, and robustness of guideline recommendations objectively and structured. Its application has only recently been introduced and has been well received as a resource to improve transparency and quality in developing clinical guidelines.

The main objective of this study was to evaluate the current maternal-fetal health GRADE guidelines contained in the International Database of GRADE Guidelines (BIGG) with the AGREE-Rex instrument, which can be adopted, adapted, and implemented by countries to improve health care.

We sought to evaluate the current maternal-fetal health GRADE guidelines in the International Database of GRADE Guidelines using the AGREE REX instrument to determine their potential to be adopted and implemented by countries to improve healthcare.

Methods

Search and selection of guidelines

In 2021, a systematic search was conducted in the International Database of GRADE Guidelines [10], specific to clinical practice and public health guidelines. This includes a list of GRADE guidelines developed worldwide [11] and classified according to the Sustainable Development Goal. The International Database of GRADE Guidelines was searched through advanced search strategies in MEDLINE/PubMed, EMBASE, LILACS, Epistemonikos, pages of guideline development groups (WHO, PAHO, National Institute for Health and Care Excellence, NICE, among other organizations), and governmental institutions in Latin America and the Caribbean [10]. In addition, a search for gray literature was performed on pages of scientific societies related to maternal-fetal health, developers' portals, and compilers of guidelines published in Spanish, French, English, and Portuguese. The searches were limited to January 2021.

Thirty clinical practice guidelines were included according to the criteria that address conditions or diseases related to maternal-fetal care. This was due to their relevance in public health impact indicators in middle- and low-income countries, which were developed with the GRADE system, and the availability of their final version with an enabled link. Guidelines classified as standard without GRADE methodology, routine care manual, or protocols were excluded from the study due to their methodological characteristics, and documents based on expert consensus, without a systematic methodological approach, or without establishing the year or scope.

In most cases, there were no discrepancies between reviewers. In those cases where differences arose, they were resolved through the intervention of a third reviewer.

The three domains (clinical applicability, values and preferences, and implementability) were selected because they are fundamental aspects that the AGREE-REX tool considers essential for assessing the quality and usefulness of clinical practice guidelines.

• Clinical applicability: this domain focuses on the relevance of guidelines in specific clinical contexts, which is crucial to ensure that recommendations are relevant and useful for health professionals in their daily practice.

• Values and preferences: this component addresses the importance of incorporating patients' preferences and values into clinical decision-making. Guidelines should be flexible and consider patients' diverse perspectives to promote a patient-centered approach.

• Implementability refers to the feasibility of applying guideline recommendations in clinical practice. Assessing this domain is essential to ensure that guidelines are theoretically correct and feasible to implement in the healthcare setting, considering resources, capabilities, and possible barriers.

• Together, these three domains provide a comprehensive assessment of the quality of guidelines, ensuring that they are scientifically sound, applicable, and practical in clinical practice.

Assessment of clinical practice guideline recommendations using AGREE-REX

The clinical practice guidelines were evaluated with the AGREE-REX tool [12], and their data was compiled in an Excel document. The AGREE-REX tool assesses the clinical credibility and implementability of practice guidelines, providing a template for development and the clinical practice guideline report to determine their grade of recommendation. The AGREE-REX is composed of 3 elements that represent nine key domains of quality of clinical practice guidelines:

1. Clinical applicability.

2. Values and preferences.

3. Implementability [13,14].

In this way, it is possible to assess the quality of the guidelines so that the experts who use them can trust them; to ensure the quality of the clinical practice guidelines by the group that prepares and disseminates them, following a structured methodology; and to evaluate these guidelines of interest by the health systems and their managers. These are essential steps for implementing and accepting recommendations based on quality guidelines.

In this search, updated guidelines were considered valid until 2021, the starting year of the present research. A data extraction form was prepared in Excel with the following variables:

1. Country of origin of the clinical practice guidelines,

2. Year of publication.

3. Original language.

4. Type of institution that developed the guide.

5. Domains of the AGREE-REX tool.

In the latter, the rigor of the recommendations contained in the clinical practice guidelines was considered.

The three key domains were evaluated separately by three peers trained in the tool to ensure the quality and homogenization of the results. In addition, a fourth team member analyzed the data resulting from these assessments. Three reviewers extracted the data in a blinded and independent manner, and a fourth reviewer did the review. None of the reviewers had been involved in developing any of the clinical practice guidelines. These reviewers included two general practitioners, a medical statistician, and a professional with a PhD in biomedical research and public health.

Each domain covered a different dimension of the quality of the guideline recommendation:

1. Clinical applicability (items 1 to 3) comprising the elements "evidence", "applicability to target users" and "applicability to patients/populations".

2. Values and preferences (items 4 to 7) comprise the elements "values and preferences of target users", "values and preferences of patients and populations", "values and preferences of decision makers and policymakers," and "values and preferences of clinical practice guideline developers".

3. Implementability (items 8 to 9) integrates the elements "purpose", referring to the implementation objectives and expected impacts of the guideline, and "local application and adoption" [15].

Each item contains item definition, quality criteria for the item, quality assessment questions (7-point response scale, where one means strongly disagree and seven means strongly agree), and suitability for use assessment questions (7-point response scale, one strongly disagree and seven strongly agree) [15].

Analysis

Using the SPSS v22 program, each domain of the clinical practice guidelines was analyzed descriptively, highlighting the percentages obtained according to the institution that developed the tool and its country of origin. Likewise, the agreement between the evaluators, the quality results of the recommendations of the clinical practice guidelines, the recommendations, and their level of evidence were measured. For this purpose, each guideline was described with variables such as country of origin of the clinical practice guidelines, year of publication, original language, and type of institution that developed the guideline. When analyzing the domains according to the institution, a Kruskal-Wallis test was performed for independent samples since the data did not have a normal distribution (Kolmogorov Smirnov test p < 0.05).

Results

A total of 1594 clinical practice guidelines were identified by title and abstract using the search methodology. Then, 144 clinical practice guidelines oriented to the maternal-fetal area were selected to evaluate the eligibility criteria. Of these, 72 guidelines that met the eligibility criteria were analyzed. The main causes of exclusion were duplication of guidelines, incomplete guidelines, or the fact that their definitive version was unavailable (Figure 1).

Figure 1

PRISMA flowchart.

CPG: clinical practice guidelines.
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Source: Prepared by the authors of this study.

Full size

Evidence mapping

Seventy-two guidelines that met the eligibility criteria for this study were analyzed. The analysis of the clinical practice guidelines by country of publication found that the United Kingdom is the country with the highest production of guidelines with 8 (11%), followed by Canada and Colombia with five respectively (7%), Spain and the United States with three each (4%), Dominican Republic with 2 (3%), and Chile, India, Italy and Peru with 1 (1%). Regarding the language of publication, English was the predominant language, with 56 clinical practice guidelines (78%) and Spanish with 16 (22%). The WHO is the leading institution in the production of clinical practice guidelines, with a total of 43 guidelines (60%), compared to governmental institutions with 19 guidelines (26%) and specialized medical societies with a total of 11 guidelines (15%).

The analysis of clinical practice guidelines by topic identified that prenatal guidelines were 7 (10%), while during pregnancy, 27 (38%) were identified, followed by postnatal (puerperium) with 13 (18%) and other maternal-fetal pathologies with 25 (35%).

Evaluation of implementability

When assessing the implementability of the guidelines with the AGREE-REX tool, an overall average of 68.49% (considered moderate quality) was found in domain 1 of clinical applicability, the domain most accepted by the reviewers. When broken down by each item, it was found that the evidence supporting the recommendations had an average of 4.9% on the response scale. An average of 5.1% was found regarding the applicability to target users. For applicability to patients/populations, an average of 4.9% was obtained.

Regarding domain 2 of values and preferences, an overall average of 49.60% was obtained (considered low quality), which the reviewers' least valued. Detailing the items individually, it could be seen that item number 4 of values and preferences of target users obtained an average of 3.9%; values and preferences of patients/population, an average of 3.79%; values and preferences of decision/policy makers, an average of 4.45%; and, finally, values and preferences of guideline developers, an average of 3.07%.

Concerning domain 3 of implementability, an overall average of 53.60% was obtained (considered moderate quality). Detailing the items individually, it became evident that, concerning the implementation objectives and expected impacts of the guideline, an average of 3.9% was obtained. Regarding local implementation and adoption of the recommendations, an average score of 4.11% was calculated.

Institution versus each domain

Domain 1 was analyzed where it was evident that, when comparing the different institutions, clinical practice guidelines belonging to the WHO complied with this domain in 78.16%, compared to specialized medical societies (32.19%) and those implemented by governmental institutions (68.11%). For a value of p = 0.000, it was shown that there is a significant difference in domain one concerning the institution that developed the instrument. Similarly, when each domain item was analyzed, there was statistical significance with a p < 0.05 (Table 1).

When analyzing domain two, we found that when comparing the different institutions, the WHO presented 54.4% (considered moderate quality) of the clinical practice guidelines that complied with this domain, compared to the specialized medical societies (24.96%) and those executed by governmental institutions (53.24%). For a value of p = 0.006, it was shown that there is a significant difference in domain two concerning the institution that developed the clinical practice guideline. Similarly, when each item of the domain was analyzed, there was statistical significance with a p < 0.05, except for the items "values and preferences of decision/policymakers" and "values and preferences of guideline developers". When each domain item was analyzed, there was statistical significance with a p < 0.05 (Table 2).

When analyzing domain 3, it was determined that 63.53% of the WHO clinical practice guidelines complied with this domain, versus those of specialized medical societies with 25.72% and those implemented by governmental institutions with 47.80%. For a value of p = 0.000, it was shown that there is a significant difference in domain 3 concerning the institution that developed the clinical practice guideline. Similarly, when each domain item was analyzed, there was statistical significance with a p < 0.05 (Table 3).

Institution versus level of recommendation of the clinical practice guideline

We proceeded to evaluate the type of institution that developed the clinical practice guideline versus the level of recommendation of the guideline (recommended when the clinical practice guideline has an average higher than 65%). A Kruskal-Wallis test was performed, obtaining a value of p = 0.007, being significant. Therefore, it can be concluded that there is a significant difference between the recommendation of a clinical practice guideline and the institution that develops it.

Country versus each domain

The country of origin of each clinical practice guideline was analyzed versus the evaluation of the AGREE-REX tool. We proceeded to evaluate, using a Kruskal-Wallis test, the domains in comparison with the origin of the clinical practice guideline to determine if there is a significant difference between the country that developed it and domain 1 of "clinical applicability". When comparing the different countries, a value of p = 0.000 was found, demonstrating a significant difference in domain one concerning the country of origin of the clinical practice guideline. When analyzing each item in the domain, there was statistical significance with a p < 0.05.

When analyzing domain 2 of "values and preferences," no significant difference was evident, having a p value > 0.84. However, when each item in the domain was analyzed, there was statistical significance only in the item "values and preferences of target users" with a p < 0.016. In the other items, there was no statistical significance. In domain 3 "Implementability", no statistical difference was detected concerning the country of origin of the clinical practice guideline with a value of p = 0.0521. This indicates that there is no difference between the country of development of the clinical practice guideline and the implementability of the guideline.

Country versus level of recommendation of the clinical practice guideline

We evaluated the country that developed the clinical practice guideline versus the level of recommendation of said instrument (its use is suggested when the guideline has an average higher than 65%). A Chi-square test was performed with a value of p = 0.143, concluding that there is no significant difference between the recommendation of a clinical practice guideline and the country that developed it (Table 4).

Subject of the clinical practice guideline

Prenatal clinical practice guidelines had an overall average evaluation of 62.23% (standard deviation 28; 53%), and other maternal-fetal pathologies, 56.4% (0.4%) (standard deviation 26; 17%). Regarding the evaluation according to the topic, they obtained a moderate quality. The variable cross-checking between the topics and the general evaluation of the clinical practice guidelines did not show a significant statistical difference.

Discussion

Maternal-fetal care is a critical area of public health that addresses the health of pregnant women and their fetuses. Maternal-fetal health care in low-income countries faces the substantial obstacle of limited access to quality care. Clinical practice guidelines are crucial in addressing this challenge by providing evidence-based strategies to optimize resource allocation, improve healthcare infrastructure, and promote efficient service delivery. In this way, accessibility for vulnerable populations is ultimately improved [16].

The findings of this study on GRADE guidelines in the maternal-fetal setting evidence the leadership of WHO as a producer of clinical practice guidelines, followed by government agencies and specialized societies. In this aspect, we identified a conglomerate of 72 maternal-fetal clinical practice guidelines between the years 2011 and 2021, which have been adopted by governmental agencies in the region’s countries, mostly with technical cooperation from PAHO/WHO. On the one hand, WHO’s leadership in producing clinical practice guidelines brings a global perspective, promoting standardized recommendations that can be implemented in various contexts. However, this centralization may run the risk that clinical practice guidelines do not adequately reflect local realities and available resources in each country, limiting their applicability [17].

The production of clinical practice guidelines that meet our inclusion criteria has been limited. WHO leads the list with discrete participation from countries such as the United Kingdom, Canada, Colombia, Spain, Italy, the United States, and Chile. Low and middle-income countries such as the Dominican Republic, Peru, and India show less production of clinical practice guidelines.

With the AGREE-REX tool, we analyzed the implementability of clinical practice guidelines from several global organizations as part of the quality assessment. Regarding the results related to the domains, the most outstanding was "applicability", while the domain "values and preferences" had the lowest score, revealing the difficulties in integrating the tools that measure this component. In general, it is observed that the set of recommendations issued by government agencies has greater adherence than those formulated by specialized societies [18].

Regarding domain elements, the following received the least favorable ratings: patient/population values, policy values, values alignment, local applicability, and resources, tools, and capacity (tables 1 and 3). Also, statistically significant higher scores were found in domains 1 and 3, with a high-moderate quality rating for recommendations made by government-supported organizations. This differs from recommendations made by professional or specialized societies that obtained a low-to-very low-quality rating, similar to those obtained by Florez et al. [19].

Differences in the average scores of the AGREE-REX tool by the type of organization and its quality are reflected in organizations supported by the government or the government itself to pursue a broader range or investments in additional methodological steps. All of these lead to higher quality scores than other developer groups, such as specialty medical societies, because they have greater resources. Clinical practice guideline developers with more financial resources and access to qualified methodologists in one way or another set higher quality and recommendation standards. As a result, there should be greater investment in higher quality guidelines and ensure their proper implementation [18].

Although our study is limited to the universe of guidelines with GRADE methodology for the analysis of the guidelines contained in the International Database of GRADE Guidelines, the utilization of the 72 clinical practice guidelines and the quality of evidence of the GRADE Guidelines clarifies the overall picture of the guidelines in terms of a worldwide maternal-fetal population. This is because, although it was the database used, it contains most of the GRADE clinical practice guidelines by specialty published to date.

On the other hand, the AGREE-REX tool is still an instrument that needs further exercise and dissemination as a resource for evaluating clinical practice guideline recommendations. The AGREE-REX could signal the personnel responsible for elaborating clinical practice guidelines, developers, and methodological groups to identify barriers and gaps that need to be closed. This, to make decisions to correct them and thus adapt and implement a clinical practice guideline with quality in the contexts where they are adapted or performed again. This allows for continuing research in other health areas with this tool.

We identified 72 maternal-fetal clinical practice guidelines from various organizations around the world, recognizing the countries' efforts in the production of these tools, as well as the incorporation in their policies of the guidelines with GRADE methodology. The evaluation with AGREE-REX showed opportunities for improvement in integrating the elements measured by domain two on "values and preferences" in the clinical practice guidelines studied. This is because there is a significant difference between the type of institution and the country of origin where the clinical practice guideline was developed in some of the domains of the tool.

The exclusion of guidelines that are not available in their final version and the temporal restriction of the search, although necessary to ensure the consistency of the data analyzed, represent some limitations of this study. The exclusion of unavailable guidelines could have limited the inclusion of emerging recommendations, which is relevant considering the rapid evolution of this field of study. On the other hand, the temporal limitation could affect the timeliness of the conclusions. These limitations underline the importance of future research integrating new guidelines.

One of the main challenges in integrating this component in maternal-fetal clinical practice guidelines is the variability and complexity of values and patient preferences in this clinical context. This makes it difficult to align them uniformly with the recommendations. In addition, local applicability and resource availability also play a key role, as differences between clinical settings and limited access to adequate resources can make it difficult to adapt guidelines to all clinical situations effectively.

In addition, the manuscript notes that organizations with government support tended to score higher in this domain because of their greater ability to perform more comprehensive methodological steps and have greater resources, allowing them to improve the quality of their guidelines. In contrast, recommendations from professional or specialized societies obtained lower scores, reflecting how the lack of financial and methodological resources can influence the quality of guidelines. This aspect is related to the research by Florez et al. [18], which also shows differences in the quality of guidelines according to the type of organization producing them.

Conclusions

Evidence-informed guidelines are fundamental resources for improving clinical practice and public health due to the rigor in their elaboration process. This study emphasizes the importance of improving the quality of maternal-fetal clinical practice guidelines developed by governmental organizations and institutions. It also highlights the need to continue with efforts to adapt or create clinical practice guidelines with GRADE methodology in different countries and evaluate their implementation. With this, it is possible to replicate documents that positively impact health indicators at a global level.

The drive towards evidence-based healthcare is a fundamental pillar for improving the quality and effectiveness of care. However, translating these guidelines into practical interventions at the local level represents one of the greatest challenges. The ability to adapt the guidelines to specific contexts, taking into account socioeconomic and cultural realities, is crucial to ensure their acceptance and effectiveness in clinical practice19.

Concerning maternal-fetal care, it is essential to highlight the need for greater inclusion and consideration of local values and preferences in developing guidelines. The production and adoption of quality clinical practice guidelines is a global challenge. There is an urgent need for broader cooperation and sharing of best practices between countries and organizations to improve the quality of care and reduce disparities in maternal-fetal health worldwide.

Regarding future lines of research, a more comprehensive review could consider other aspects of health care in low-income countries, such as child health, obstetric care in general, infectious diseases, and public health interventions. To obtain a more complete picture of the situation, future research could broaden the focus to encompass a wider range of medical issues and practices. This would identify areas for improvement in health care and promote greater impact on overall health.