Potential for Conflict of Interest in the Evaluation of Suspected Adverse Drug Reactions: Title and subTitle BreakA Counterpoint

Health care practitioners and patients seek safe and effective drugs. However, no drug is completely safe; all drugs have toxic effects that must be balanced with their benefits in deciding whether they should be marketed or used in any given person. To inform such decisions, the United States relies on a drug approval system whereby preclinical studies precede 3 phases of clinical studies. Collectively, these usually include 500 to 3000 exposed patients and 2 or more confirmatory trials, demonstrating before marketing that a drug is effective and reasonably safe for its recommended use.¹ Thus, adverse reactions occurring in 1% or more of exposed patients are usually well described upon marketing. However, rarer adverse reactions are not well characterized until after marketing.² This reflects a deliberate societal decision to balance delays in access to new drugs with delays in information about rare adverse reactions. To provide the missing information, the United States maintains a postmarketing surveillance system including passive collection of spontaneous reports of adverse drug reactions (ADRs) to generate signals of possible adverse drug events. This is supplemented by formal pharmacoepidemiology studies testing those hypotheses, confirming or disproving potential signals from the spontaneous reporting system (SRS).²

Cerivastatin uniquely challenged this system. It is now well known that cerivastatin, when combined with gemfibrozil, poses an increased risk of rhabdomyolysis. In fact, a rhabdomyolysis warning was included in the original cerivastatin label upon its US launch, along with a warning against use with fibrates, based on experiences with previously released statins. Postmarketing reports of rhabdomyolysis soon after marketing were therefore neither unexpected nor alarming. Thus, the normally difficult decision to act more aggressively on spontaneous reports, always subjective, was even more difficult here. I share the concern by Psaty et al³ that drug manufacturers have an inherent conflict of interest in making such decisions. However, cerivastatin presents a unique opportunity to highlight SRS limitations, and its potential misuse, and provides insights into how to improve the US drug monitoring program. This commentary will not consider issues of the ongoing litigation, including manufacturer conduct, timelines of company memos, the litany of legal exhibits, or the accusations made by Psaty et al regarding those issues. Rather, it will discuss the scientific and policy issues raised by Psaty et al, describe the SRS in some detail, and discuss appropriate uses of SRS data and organizational implications of its limitations. Cerivastatin serves as an instructive example. The company response addresses the cerivastatin legal issues elsewhere in this issue.

Description. The identification of possible ADRs after marketing relies on spontaneous voluntary reporting to industry or regulators of reactions observed in clinical practice, fundamentally a 1950s-era approach.² The Food and Drug Administration (FDA) receives approximately 280 000 such reports annually, consolidating them into a large database.² Few reports (<10%) are submitted directly to the FDA by health care practitioners and consumers. Rather, manufacturers receive more than 90% of reports and must then process, analyze, and report to the FDA all ADRs reported to them after use of their products.

A major advantage of the SRS is that it incorporates all drugs, prescribers, dispensers, and patients, casting the broadest possible net to capture events. However, this system is subject to underascertainment (not recognizing an event is due to a drug) and overascertainment (erroneously ascribing an adverse event to a drug). Furthermore, it is subject to vast underreporting, with published reporting rates ranging from substantially below 1%⁴ to 53%,⁵ and variations depending on event severity and acuteness, for example. Reporting is somewhat more complete for newer or more recently marketed drugs than older drugs. External events (eg, letters to the editor in a medical journal or lay newspaper, increased suspicion about a drug’s risk, or “Dear Doctor” letters) can easily modify ascertainment or reporting rates.

To calculate true ADR rates, one needs an accurate number of events in the exposed population (numerator) and an accurate number of exposed individuals (denominator). The SRS has neither. There is no true numerator because it is unclear how many ADRs actually occurred in a specific population. Accurate denominators likewise are unavailable; sponsors and the FDA know the sales of a given drug but often lack the number of individuals exposed over time. Furthermore, sponsors and the FDA do not know the number of people who used the drug and whose ADRs would have been reported had they experienced such an event, which would be the correct denominator for a numerator consisting of only reported cases.

The SRS focuses on detecting unknown adverse effects from individual new drugs; consequently, it is not adept at identifying medical adversity from recognized ADRs due to drug misuse, underprescribing, overprescribing, and other medication errors, although these affect public health more than previously unidentified rare adverse effects.² Nevertheless, this system remains the primary and best method for identifying ADRs to newly marketed drugs.

Appropriate Use of SRS Data. To quote the FDA, “Because of these limitations, AE [ADR] reports are primarily useful for hypothesis generating, rather than hypothesis testing.”⁶ Indeed, the SRS generates numerous hypotheses, many of which are neither investigated further nor confirmed. To transmit information about these untested hypotheses to patients or health care practitioners would be extremely misleading, as many are untrue. This contrasts sharply with the clinical trials referred to by Psaty et al,³ where recently there have been calls for complete public disclosure of results of all such trials. I agree and would expand this call to include results of all pharmacoepidemiology studies. However, such trials and studies are designed to test hypotheses, which is not the purpose of the SRS.

In analyzing ADR data, Psaty et al³ used a measure increasingly utilized by regulators in pharmacoepidemiology. The authors estimated a “relative reporting ratio,” analogous to the more commonly used term proportional reporting ratio (PRR), which is the relative frequency of reports of a specific event compared with all events reported for a specific drug, divided by the corresponding quantity for other drugs. These “disproportionality measures” were developed to help identify signals to be explored in subsequent controlled studies and have recently been reviewed in the field’s journal.^{7 - 14} However, these exercises are formal statistical analyses of poor, incomplete, and biased data (ie, spontaneously reported adverse reactions). No matter how sophisticated, analyses of such data can readily be misleading. The controversial nature of these measures stems from the underlying data—spontaneous ADR reports—which are “only the suspicions of doctors and clinicians.”⁷ The various biases that affect such data can easily affect different drugs differently, leading to unequal reporting and incorrect results if SRS data are used to compare among drugs. More sophisticated analyses cannot correct for the huge imperfections in the data source. Pharmacoepidemiologists hope that using PRRs will allow for earlier hypothesis generation, although this goal still remains to be evaluated rigorously. The major role of PRRs, though, remains hypothesis generation. Indeed, some pharmacoepidemiologists question whether disproportionality analyses ever merit publication, as “ . . . anecdotal case reports and disproportionality measures of them are of the same essence, and distinct from controlled epidemiologic studies.”⁸

If one cannot draw conclusions from PRRs, what is the appropriate response when a “signal” of a possible problem is detected? With rare exceptions, the appropriate response is further investigation via formal epidemiological study. Deciding if and when a signal requires such study is, however, subjective. As Psaty et al note, a manufacturer has an inherent conflict of interest in making such a judgment. This underlies their concern.

The events surrounding cerivastatin are instructive. As Psaty et al indicate, a signal emerged from the SRS. One could argue whether the signal was responded to quickly enough, but neither these authors nor I were part of that decision process. Although “hindsight is 20/20,” such decisions can be hard to make while events are still unfolding. However, appropriate responses did occur, including labeling changes, “Dear Doctor” letters, and ultimately drug withdrawal. If the manufacturer had succeeded in eliminating the concurrent use of cerivastatin and gemfibrozil, as it attempted, the number of affected patients likely would have been dramatically smaller. The manufacturer also launched a formal pharmacoepidemiology study,¹⁵ presented and published in abstract form in mid-2002, confirming in a large managed care database an increased risk associated with cerivastatin when combined with gemfibrozil. The FDA, accessing the same SRS data, conducted its own analysis,¹⁶ similar to Psaty et al. Describing the market withdrawal of cerivastatin in a 2002 letter to the editor, the FDA authors acknowledged, “Rigorous comparisons between drugs that are based on these data are not recommended, since many factors can affect reporting and an unknown number of cases may not be attributed to the drug or reported to the FDA. Reporting rates are not incidence rates.”¹⁶ The FDA also launched its own formal pharmacoepidemiology comparative study using 11 managed care databases. Presented and published in abstract form in mid-2004, the study indicated increased risks with the use of both gemfibrozil and cerivastatin, and a marked synergy between them.¹⁷

What lessons can be drawn from these experiences? First, patients and health care practitioners must recognize that regulatory approval does not guarantee safety. By design, premarketing studies are limited in size and duration; postmarketing monitoring remains critical. Second, new drugs must be tracked closely following marketing to identify any new adverse effects previously undetected. This responsibility currently falls mainly to the drug’s manufacturer. Third, given manufacturers’ inherent conflicts of interest, adequate oversight of their decisions is necessary. Such responsibility falls to the FDA, recently strengthened in this regard with resources from the 2002 reauthorization of the Prescription Drug User Fee Act.²

Do critical gaps remain? Unfortunately, they do. Currently, attention focuses on whether to approve (or withdraw) new drugs. No organization primarily focuses on monitoring effects of older drugs.² Yet it is well recognized that old drugs, used poorly, are responsible for most public health damage caused by ADRs.² Furthermore, no organization is formally responsible for developing new methods for pharmacoepidemiology (eg, faster and more systematic approaches to hypothesis generation), evaluating new methods for pharmacoepidemiology research (like PRRs), conducting pharmacoepidemiology studies for which results cannot be suppressed, training scientists in disciplines needed to conduct such research, or educating health care practitioners and recipients of prescription drugs about their effects. No organization is charged with formalizing and testing decision rules for when signals from the SRS should be acted on, and how. Analogously, no organization is charged with developing and testing new methods for improving physicians’ use of drugs in clinical practice.

The closest the United States has to such an organization are the 7 Centers for Education and Research in Therapeutics (CERTs), based in academic centers and funded by the Agency for Healthcare Research and Quality (AHRQ).¹⁸ These activities are central to the mission of the CERTs. However, the fiscal year 2004 budget for the entire CERTs program is $5.8 million (L. Bosco, MD, written communication, AHRQ, August 2004). This is meager when contrasted with the more than $800 million spent developing a single new drug,¹⁹ the $33.2 billion pharmaceutical industry expenditure in 2003 on research and development,¹⁹ the $15.7 billion pharmaceutical industry expenditure on promotion in 2000,²⁰ or the more than $40 billion expected to be spent annually on the new Medicare drug benefit.²¹ The Joint Commission on Prescription Drug Use, triggered by Senator Edward Kennedy, called for creating a private nonprofit Center for Drug Surveillance to address these gaps; that call came in 1980.²² Twenty-four years later, I join Psaty et al in renewing that call. This could be accomplished by vastly increasing the number of CERTs and the funding of each. It also could be accomplished by forming a new organization. Regardless, such investment is critical to optimizing the health outcomes resulting from the substantial sums spent in the United States on therapeutics.

The United States spends $122 billion annually on pharmaceuticals,²³ and some researchers place fatal ADRs between the fourth and sixth leading causes of death.²⁴ According to a 1990 study by the US General Accounting Office, 51% of approved drugs have serious adverse effects undetected before approval.²⁵ All patients and physicians seek safer drugs, and safer drug usage. Yet the resources expended to ensure drug safety are extraordinarily limited. The central problem is not rare adverse reactions to new drugs, albeit these are what attract attention, resources, and litigation. Although hindsight always raises questions about whether a problem could have been detected earlier, the events surrounding cerivastatin serve as a clear example of how the system should work. Although rhabdomyolysis was already an established adverse effect of statins, a signal was seen that the rate might be even higher with this statin, and actions were taken, informed by a formal pharmacoepidemiology study mounted early by the manufacturer and, later, a larger study by the FDA. However, there is indeed a conflict of interest in asking industry to monitor its own drugs.

It is imperative, therefore, that the FDA continue to be bolstered, both in its efforts to monitor drug safety and in its new risk management initiatives.² In my opinion, there is no need for additional duplicative regulatory oversight of newly marketed drugs; our FDA colleagues can be trusted to do the job, given sufficient resources. However, it is also critical that the nation commit to adequately funding CERTs, a Center for Drug Surveillance, or some analogous structure, to complement the regulatory mission of the FDA. This would significantly enhance the capacity of the field to respond to such concerns and improve the speed and quality of that response.