Phenomena of Retraction: Title and subTitle BreakReasons for Retraction and Citations to the Publications FREE

VARIOUS FACTORS can lead to the retraction of a publication. There have been some notorious cases of scientific misconduct in recent years. LaFollette¹ and Whitley et al² report on some of the more widely publicized cases, and a collection of essays edited by Lock and Wells³ addresses generally the matters of fraud and misconduct. While these concerns are legitimate and pressing, the biomedical literature is also affected by error that can render the reported results of research useless at best and dangerous at worst. The work done by Stewart and Feder⁴ focuses on the occurrence of error. Awareness of the retraction and reasons for retraction might affect the frequency with which such articles are cited subsequent to retraction.⁵

Based on the available background research, a set of expectations can be stated: the authors are the ones doing the retracting; most retractions occur because of scientific misconduct or unavoidable error; the entire article should be regarded as an invalid scientific article; and retracted publications continue to be cited after retraction.

The data for analysis came from a MEDLINE search (1966-August 1997) that used the publication type "retraction of publication." This strategy includes 2 limitations: an article must be formally retracted to be assigned this publication type, and indexers must recognize the retraction and assign the publication type. Further, this study focuses entirely on retraction, so corrections and errata are not included. The search yielded 235 retracted articles. This population is a very small subset of the MEDLINE database, but it does include all publications formally identified as retractions. The next task was to classify the articles according to the following characteristics: who retracted the publication; what content was retracted; why the article (or portion of the article) was retracted; and how long after publication the retraction occurred.

To determine citing activity, Science Citation Index was searched to identify all citations to each of the retracted publications. A 1-year period after publication of the retraction was inserted before a citation was considered as postretraction to allow for indexing of the retraction to be in place. Searchers of the MEDLINE database would then have ready access to the retraction statement. Also, the 1-year period compensates for publication lag; that is, if a manuscript is in press at the time a retraction statement is published, then that article could, in good faith, contain citations to retracted items. This study focuses on postretraction citations appearing in journals indexed in the Abridged Index Medicus (AIM), since that source contains the most clinically relevant journals. The postretraction citations were divided into 3 categories: the citing article acknowledged the retraction, the citing article explicitly cited the retracted article as presenting valid research, or the citing article implicitly cited the retracted article as valid. In addition to noting the category of citation, the kind of citing publication (letter, review article, or article) and where in the citing article (introduction, methods, results, discussion, or conclusions) the citation occurs were recorded.

One element of the phenomenon of retraction addressed was the length of time between publication of an article and its retraction. The mean time from publication to retraction was 28 months. This mean includes the retractions by 1 author of 4 articles 10 years after their publication. Controlling for this anomalous case yields a mean time from publication to retraction of 25.8 months (range, 2-197 months).

One or more of the authors retracted 190 of the 235 articles; 45 were retracted by others, including institutional investigating committees or deans, journal editors, or legal counsels. Retraction of 91 articles occurred because of some kind of error. The categories of error identified were the following: error in the methods or analysis in 23, problems with the data in 37, and problems with the sample (such as contamination) in 31. A total of 86 articles were retracted because of misconduct or presumed misconduct. Retraction was classified as being due to misconduct only if the statement of retraction clearly admits to wrongdoing on the part of one or more of the authors. Presumed misconduct refers to those instances where one or more of the authors raises serious questions about the efficacy of the work done by other authors. An additional 38 articles were retracted because the author(s) could not replicate the results. These were categorized separately because it is not possible to determine if the results were not replicable because of unavoidable error or because of some wrongdoing. The remaining 20 articles were retracted for other unclassifiable reasons. In 3 instances the retractors were unclear as to reason, in 3 cases idiosyncratic reasons were given, and in 14 cases no reason was given. In 200 instances the entire article was retracted; in 35 cases part, but not all, of the article was retracted.

The 235 articles received a total of 2034 postretraction citations. Of these citations, 299 appeared in journals indexed in AIM. The majority of the citations were found in articles reporting research or clinical practice (n=277). Only 14 citations appeared in letters, and 5 of those letters referred to the retraction. Review articles accounted for 8 citations, and only 1 of those acknowledged the retraction. Of the 299 postretraction citations appearing in AIM journals, 19 acknowledged the retraction in some way. Given that 5 of these acknowledgments appeared in letters and 1 appeared in a review article, only 13 articles reporting research made specific mention of the retraction. Of the remaining citations, 17 explicitly treated the retracted article as valid, usually by naming the authors of the article or mentioning specific elements of their findings or methods. The remaining 263 citations include implicit approval of the retracted work, usually in the form of brief mention or bibliographic reference in a passage that in no way questions the validity of the research.

The second of the stated expectations proved to be somewhat problematic; the first, third, and fourth were shown to be, with a few exceptions, supported. The results of this study strongly indicate that retraction of a publication, even though the retraction may be visible in the journal and is clearly noted in the MEDLINE database, does not ensure that all subsequent researchers will be alerted to the retraction and will cease making reference to the retracted work. It seems to matter little if the cause of retraction is error or misconduct; citations to any retracted article may well continue.

While 263 of the citing articles in this study embody implicit positive citation to retracted publications, and while these citations tend to appear in the introduction (n=117) or discussion (n=153) sections of articles, the citations still ensure that the retracted articles continue to appear in citation indexes and that they may be retrieved by readers of the citing article. (It should be noted that 47 citations appear in the methods section, 6 in the results, and 2 in the conclusion.) If a researcher comes upon one of the citing articles and finds the work done there of use, then he or she may turn to cited works and incorporate them into his or her work as though the work were valid. Such a researcher, who does not retrieve the information through a formally structured MEDLINE search, may be unaware of the retraction.

It should be pointed out that biomedical science tends to be self-correcting; that is, work that is not replicable because of error or misconduct is usually dismissed in time. However, there may be a great deal of time, effort, and money spent in discovering that some research is not useful. If erroneous or fraudulent work lives on in the literature, the amount of time, effort, and money to correct work may be even greater. It should also be noted that this study focuses on work that has been acknowledged to be based on error or misconduct. The question remains as to how much erroneous or fraudulent work goes undetected or unacknowledged. That is a larger question that should be addressed by the biomedical community; the evidence provided by this study suggests that it is a serious question with profound implications both for research and for clinical practice.