Questioning the value of metastudies
|I am going to post excerpts from one article on two different medicalskeptic posts because of the importance of the two subjects brought up in the article. The first subject is the value of meta-analysis in increasing knowledge. The author provides several very important caveats.
The link is to the blog Evidence in Medicine and written by Dean Rindy
hierarchies of evidence are listed for the EBM world, meta-analyses of randomized trials generally sit at the pinnacle.
And yet, the actual meta-analyses that you encounter when researching a clinical question can be far less enlightening. Even if we grant a pass to the many systematic reviews at The Cochrane Collaboration that conclude with the a priori obvious fact that no high quality RCTs addressing a question have been performed, and another pass to the reviews that find a single RCT and publish its results as the results of the systematic review, we are still left with the innumerable meta-analyses that seem to provide less of a window on truth than the underlying trials.
Frequently such meta-analyses are either driven by the single large RCT that everyone would have cited anyway or, worse, a number of small, poorly-performed RCTs are combined with a moderate-sized, well-performed RCT and alter the results away from what was likely the best estimate of reality: the results of the well-performed RCT.
Meta-analysts often seem to either be too removed from their subject area and thus lack the expertise to really understand what went clinically right and wrong in the underlying RCTs (or be unwilling to use that knowledge to discriminate among the trials), or be too cozy with a single trial (typically as an author) and thus too willing to ding trials that found conflicting results.
Ultimately, meta-analysis only rarely seems to importantly advance our knowledge of an issue beyond where we would have found ourselves by just reading through the RCTs…..” more http://tiny.cc/QW0K9