Comparing Results From Meta-analyses vs Large Trials

To the Editor.—In their review and discussion of relevant issues in comparisons between meta-analyses and large randomized controlled trials,¹ Ioannidis and colleagues propose to explain why our external comparisons² (comparing large randomized controlled trials with previous meta-analyses) might produce larger differences than the internal comparisons used by Villar et al³ and Capelleri et al⁴ (they recalculated the results of each meta-analysis after removing the largest trial and then compared these results with those of the large trial that had been removed). However, we do not think the Villar-Capelleri protocol is a valid method of testing the extent of concordance between meta-analyses and large randomized controlled trials because it has an important flaw. It does not test the meta-analysis as it was published (the issue of interest) but a new version, with a different result, which excluded the large trial and only included small trials in the analysis. The meaning and representativeness of this new version is uncertain. Furthermore, if the method was generally adopted in future meta-analyses it would lead to paradoxical results. If the large trial included in a meta-analysis is going to become the "gold standard," then the meta-analysis will become superfluous. It would be logical to use the large trial as the best evidence⁵ and forget about the meta-analyses.