Tag Archive for: AMSTAR 2

You are here: / AMSTAR 2

Posts

Reaching for the stars – rating the quality of systematic reviews with the Assessment of Multiple Systematic Reviews (AMSTAR) 2

/by

The number of published systematic reviews and meta‐analyses in the urological literature has dramatically increased in recent years [1]. This is good news given their importance in guiding clinical decision‐making, guideline development and health policy. However, many of these studies are of low quality, raising concerns about the trustworthiness of their results. As with other research studies, it is therefore important for readers to have a framework for determining the quality of a given systematic review. Therefore, in 2017 BJU International launched a scoring system for systematic reviews that provides readers with a summary assessment as to whether established methodological safeguards against bias for systematic reviews have been met [2]. This is based on the Assessment of Multiple Systematic Reviews (AMSTAR), a validated instrument that assesses methodological quality on an 11‐point scale (0–11), with higher scores reflecting greater methodological rigor and all criteria being given the same relative weight [3].

Recently, an updated version of this instrument has become available, offering a better assessment of systematic reviews [4]. The revised instrument (AMSTAR 2) includes 10 of the original domains; it has 16 items in total (compared with 11 in the original), simpler response categories to the original AMSTAR, and provides an overall rating that is largely based on seven critical domains that should all be met. These relate to: (i) documentation of an a priori registered protocol in Prospective Register of Systematic Reviews (PROSPERO) or through Cochrane, (ii) a comprehensive literature search, (iii) explicit justification for excluding studies, (iv) a risk of bias assessment of included studies, (v) appropriate use of meta‐analytical methods, (vi) consideration of risk of bias when interpreting the results of the review, and (vii) assessment of presence and likely impact of publication bias. Other, non‐critical domains include a clear description of the study question in Population, Intervention, Comparison, Outcome (PICO) format, study selection and data extraction in duplicate, and identification of sources of funding of the studies included in the review and the review itself. This results in a four‐tiered rating (high, moderate, low, and critically low) that reflects the confidence that a reader may place in the results. Notably, a high‐quality rating requires no critical weakness and allows for only one non‐critical weakness. More than one non‐critical weakness drops the rating down to moderate, and just one critical weakness (such as lack of an a priori protocol) drops the rating down to low. Any review that has more than one critical weakness will be rated as critically low.

BJU International editors will routinely apply this AMSTAR 2‐based scoring system to screen for methodological quality in order to raise the awareness of this issue and promote reviews of higher quality (Fig. 1)[1]. Needless to say, BJU International is not the place for systematic reviews of sub‐optimal methodological quality in which the readers cannot place their trust. Meanwhile, we also fully understand that methodological rigor is not everything but has to be paired with clinical relevance and newsworthiness. Much has been written about the dramatic redundancy of systematic reviews on the same topic; in certain areas of medicine, the number of systematic reviews exceeds that of eligible studies that these reviews included [5]. Therefore, when systematic reviews already exist, there needs to be a clear rationale for any ‘encore’ performance. BJU International also encourages the development of systematic reviews by author teams that are financially unconflicted and have thoughtfully managed any intellectual conflict of interest.

Figure 1: New BJUI rating system of systematic reviews based on AMSTAR 2. The number of coloured stars in the inner and outer layers of the system represents completeness of an individual critical domain and overall confidence rating of the systematic review, respectively. The number in the middle of the system refers to the summary AMSTAR 2 score based on the overall confidence rating of the systematic review (high: 4, moderate: 3, low: 2, critically low: 1).

Through this initiative, BJU International not only intends to become the premier journal for high‐quality systematic reviews as they relate to urology, but also to move the field forward, reducing redundancy and waste. As we embrace the higher standards of AMSTAR 2, we present the first review to be scored using this method in this issue [6] and we encourage all systematic review authors to accept this challenge and reach with us for the stars.

References

  1. Han JL, Gandhi S, Bockoven CG, Narayan VM, Dahm P. The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality. BJU Int 2017; 119: 638–49
  2. Dahm P. Raising the bar for systematic reviews with Assessment of Multiple Systematic Reviews (AMSTAR). BJU Int 2017; 119: 193
  3. Shea BJ, Grimshaw JM, Wells GA et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007; 7: 10

 

About the authors:

Dr Philipp Dahm is Professor of Urology and Vice Chair of Veterans Affairs at the University of Minnesota. He also serves as Director of Research and Education for Surgical Services at the Minneapolis Veterans Administration Medical Center (@EBMUrology).

 

Dr Jae Hung Jung is from the Department of Urology, Wonju College of Medicine, Yonsei University, Korea.

 

 

 

© 2024 BJU International. All Rights Reserved.