Assessment of GRADE utilisation in published systematic review and meta-analysis

INTRODUCTION Systematic reviews are widely regarded as the cornerstone of evidence-based practice, offering comprehensive summaries of existing research to inform healthcare decisions.[1] However, the reliability of conclusions drawn from these reviews hinges not only on the quality of the included studies but also on the transparency and rigour with which evidence is synthesised and evaluated.[2] The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach has emerged as the internationally accepted framework for rating the certainty of evidence and the strength of recommendations in systematic reviews and clinical guidelines.[3,4] GRADE provides a structured and transparent process to evaluate critical domains such as risk of bias, inconsistency, indirectness, imprecision, and publication bias, thereby guiding users in interpreting the robustness of effect estimates.[5] Despite widespread endorsement by institutions such as the Cochrane Collaboration and the World Health Organisation, the actual use, correct application, and reporting quality of GRADE in published systematic reviews remain inconsistent. This study aims to assess how systematically and appropriately the GRADE approach is utilised in published systematic reviews. By identifying patterns, gaps, and common methodological limitations in the application of GRADE, this assessment seeks to inform the best practices and enhance the quality and reliability of future evidence syntheses. METHODS A systematic search was conducted to identify published systematic reviews and meta-analyses (SRMAs) in the Indian Journal of Anaesthesia (IJA) from 1 July 2024 to 30 June 2025 that reported the use of the GRADE approach. The search was limited to articles published in the main issues of the journal during this period. All retrieved articles were screened independently by two reviewers, with any disagreements resolved through consensus with a third reviewer. Titles and abstracts were initially screened to assess eligibility, followed by full-text review to confirm the application of the GRADE methodology. Data were extracted using a pre-designed proforma and analysed descriptively. The key parameters assessed included the proportion of SRMAs applying the GRADE approach, the domains reported (risk of bias, inconsistency, indirectness, imprecision, and publication bias), the level of certainty assigned to outcomes (high, moderate, low, or very low), and the nature of recommendations made (strong or conditional) [Supplementary Table 1].Supplementary Table 1: Assessment of evidence profile after GRADE analysisAll analyses were conducted using Excel. Results were reported as frequencies, percentages, and tabulated summaries. RESULTS A total of 22 studies were included in the assessment [Table 1], comprising 20 systematic reviews with meta-analyses (SRMAs) and two network meta-analyses (NMAs).[6-27] The GRADE approach was applied in 16 studies (72.7%) to evaluate the certainty of evidence [Table 1].[6,8-22] Across these 16 studies, 80 outcomes were assessed using GRADE. Of these, 11% (9/80) were rated as high-certainty evidence,[11,13,18,21] 30% (24/80) as moderate,[6,11-16,19,20] 35% (28/80) as low,[6,8-10,11-16,18-22] and 24% (19/80) as very low-certainty evidence.[6,8-10,11,17,18,20,21,22] Notably, none of the studies made strong recommendations. Only two studies provided conditional recommendations.[9,14] Additionally, seven studies offered suggestions based on the GRADE assessment,[8,11,18-22] and four studies explicitly highlighted the need for further research [Table 2 and Supplementary Table 1].[10,15-17]Table 1: Study characteristicsTable 2: Study outcomesDISCUSSION The GRADE approach provides a systematic framework for rating the certainty of evidence and formulating recommendations in systematic reviews and guidelines.[1] It evaluates the confidence in effect estimates based on five core domains: risk of bias, inconsistency, indirectness, imprecision, and publication bias.[1,3] Additional domains, like magnitude of effect, dose-response gradient, and residual confounding, can lead to upgrading or downgrading in specific scenarios.[28] Despite its methodological rigour and wide adoption in evidence-based practice, our analysis highlights certain gaps and inconsistencies in its use across published SRMAs. In our assessment of 22 SRMAs and NMAs, only 16 studies (72.7%) applied the GRADE framework to assess certainty of evidence. Among the 80 outcomes evaluated through GRADE, the majority were rated as low or very low certainty (35% and 24%, respectively), with only 11% of outcomes deemed to be of high-certainty evidence. This suggests that while GRADE is being increasingly adopted, the underlying evidence base in these reviews remains limited in quality—often due to methodological flaws in primary studies, inconsistency of results, or imprecision in estimates. GRADE requires structured, transparent judgments, and the rationale for downgrading or upgrading should be explicitly documented.[29] However, several reviews lacked clarity in justifying their certainty ratings. For instance, the domain of imprecision was inconsistently interpreted and often without reference to optimal information size (OIS) or confidence interval width. Similarly, inconsistency was sometimes downgraded without quantifying heterogeneity using I² statistics or visual inspection of forest plots.[30] Another critical gap was the disconnection between certainty ratings and the formulation of recommendations. While GRADE encourages integrating the evidence profile with values, preferences, and clinical context to form strong or conditional recommendations, only two studies offered conditional recommendations, and none provided strong recommendations.[9,14] This limits the practical utility of the reviews in guiding clinical decision-making or policy formulation. Several studies merely described findings narratively without progressing to formal GRADE-based guidance.[7,23-27] Finally, the domain of publication bias was infrequently addressed, even when GRADE was used. Despite tools like funnel plots or Egger’s tests being available, many reviews did not report whether this domain was considered, possibly due to small numbers of included studies or a lack of methodological expertise.[13] CONCLUSION The application of the GRADE approach in SRMAs and NMAs remains inconsistent, with most studies reporting low to very low certainty of evidence and few translating findings into actionable recommendations. Strengthening the use of GRADE through standardised application and transparent reporting can enhance the reliability and clinical utility of evidence syntheses. Presentation at conferences/CMEs and abstract publication None. Study data availability De-identified data may be requested with reasonable justification from the authors (email to the corresponding author) and shall be shared upon request. Disclosure of use of artificial intelligence (AI)-assistive or generative tools The AI tools or language models (LLM) have not been utilised in the manuscript, except that software has been used for grammar corrections. Declaration of use of permitted tools The scales, scores, figures, and tables are freely available and not copyrighted. Authors contributions AS: Conceptualisation, Designing, Definition of intellectual content, literature search, data acquisition, data analysis. AL: Conceptualisation, Designing, Definition of intellectual content, literature search, editing, review and approval. KS: Definition of intellectual content, data acquisition, Data analysis, manuscript preparation, and editing. IM: Concepts, Design, Definition of intellectual content, literature search, editing, review and approval. Supplementary material This article has supplementary material and can be accessed at this link. Supplementary Material at https://links.lww.com/IJOA/A59. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.