Why Guideline-Making Requires Reform

Guidelines are a constructive response to the reality that the practicing physician requires assistance to assimilate and apply the exponentially expanding, often contradictory, body of medical knowledge. Guidelines are widely perceived as evidence based, not authority based, and therefore as unbiased and valid. Because they are sponsored by organizations, staffed by experts, and conducted according to apparently formal processes, the products of the exercise—the guidelines—are generally assumed to have the same level of certainty and security as conclusions generated by the conventional scientific method. For many clinicians, guidelines have become the final arbiters of care.

Guidelines have taken hold and multiplied. The National Guideline Clearinghouse has registered 2373 guidelines produced by 285 organizations.¹ Indeed, any group of individuals can designate itself a guideline group and different guideline groups have reviewed the same disease and reached different conclusions.^{2 - 3} If the process is so secure, how is this possible? After all, replication is the distinguishing characteristic of scientific knowledge and an essential test of the validity of any scientific statement.

Given the influence of guidelines on clinical practice and given the fact that the process has been, and remains, essentially unregulated, the guideline process deserves review. In this Commentary, we examine the sources of guideline authority; identify major limitations of the present process; briefly address the issue of conflict of interest, both for the individuals who staff the committees and the organizations that govern them; and provide suggestions for reform that may help improve the conduct of the process. Examples are principally selected from lipid guidelines because many clinicians are familiar with them and they illustrate issues that apply to many other guidelines.

The anchoring authority of the guideline process is the belief that guidelines are evidence based, not opinion based, and therefore their conclusions flow directly from the conclusions of studies. Accordingly, the outcome is perceived to be impersonal and inevitable. Guidelines are issued under the imprimatur of the organization that sponsored them. This immersion of a specific product within the aura and prestige of an overall organization is another source of their authority. Guidelines also acquire authority from the prestige of the journals in which they are published. Moreover, guideline decisions are characteristically unanimous and the single voice in which guidelines are expressed undoubtedly adds enormously to their influence.

Typically, guideline committees are created by professional organizations committed to the care of patients with one disease or group of diseases. These organizations establish the mandate for the guideline group, select its members, and provide powerful mechanisms for the dissemination of the products of the process. Once issued, the sponsoring associations become promoters and defenders of the guideline that has been produced. Few associations submit the final products of the guideline process for external review before they are accepted and, therefore, in a limited but real sense, the committee, which is a creation of the organization, becomes the final arbiter of its process.

Foundational principles and operating procedures have been suggested for the guideline process.^{4 - 5} Nevertheless, most committees have considerable latitude to establish their own working rules. For example, there do not appear to be explicit rules as to the range of expertise that must be included within the committee. Epidemiologists and economists are often minimally represented. Different topics require different repertoires of talents. Importantly, even when it is known that areas of legitimate controversy will be covered, there is often no attempt to ensure that all sides will have reasonable opportunity to present their evaluation of the evidence and participate in the decision-making process.

Although guidelines strive to be evidence based, they cannot be derived strictly, solely, and incontestably from the evidence. On many, if not most, issues the evidence, no matter how extensive, remains incomplete. For example, even though the general conclusion that statins reduce the risk of vascular events is incontestable, many specifics remain debatable, such as how intensive therapy should be, when therapy should be started for different clinical situations, and which markers should be used to monitor therapy. Because gaps in the evidence are inevitable, they must be filled in with judgments, and judgments tend to preserve previous positions. Thus, what is to be decided is often already decided with the selection of the deciders.

Unanimity is not a natural component of science. Given the number and complexity of issues reviewed and given that scientific knowledge is at any moment incomplete, unanimity is obviously a tactic, not a necessary result. Debate may have been brisk within the committee but usually all evidence has been expunged from the final document. Contrast the guidelines with the decisions of any court of appeal in which some judgments are issued unanimously but most are not. Most decisions are divided, with reasoned argument recorded by those on either side, often with different analyses by different dissenters. These minority opinions, not infrequently, provide the legal scaffolding for future reversal of the majority decision they opposed.

Although unanimity is the rule in individual guidelines, it can be strikingly absent when different guidelines are compared. The debate as to whether low-density lipoprotein cholesterol (LDL-C) or apolipoprotein B (apoB) is a more powerful marker of the risk of vascular disease illustrates that guideline groups may not just disagree—they actually may contradict each other. For instance, in the past 6 months, 4 reports have compared LDL-C and apoB, with 2 supporting LDL-C over apoB^{6 - 7} and 2 in favor of apoB for predicting cardiovascular risk.^{8 - 9} The 2 reports that favor LDL-C state categorically that there is no published evidence allowing apoB treatment targets to be established. The 2 that chose apoB cite multiple studies supporting their position in favor of an apoB target. Only one⁹ presents a complete, detailed, organized review and analysis of the evidence including the technical accuracy and reproducibility of the 2 measures. The discordance between the views on apoB vs LDL-C is disconcerting, but not surprising given the failure to even agree on what constitutes evidence or how that evidence should be graded.

Guidelines generally are outside the accepted procedures of scientific publication in which acceptance for publication is the independent decision of the editorial staff of the journal and that decision must include fair independent review. Not only do the scientific organizations that commission guidelines usually not subject the guideline reports to independent review before they issue them, the journals that those organizations control often must publish their guidelines essentially as is. Thus, one of the core processes of science—the necessity to submit any analysis to the independent review of others—is bypassed. The process can be bypassed, but not the consequences, because review is an essential and constructive element in the never-ending effort to root out error.

By favoring one test over another, or one therapy over another, guidelines often create commercial winners and losers, who cannot be disinterested in the result and who therefore must be separated from the process. While the groups that finance medical care do not automatically accept these recommendations, they undoubtedly have a major influence on their decisions. Accordingly, those who write the guidelines and those who issue them should be free from significant conflict of interest. This issue has received attention in the past,^{10 - 11} but attention has not ensured coherent, comprehensive action.

There have been constructive, legitimate relationships between industry and academic medicine, both with regard to education and research. However, there also have been inappropriate and intolerable relationships, which escaped attention for much too long.^{12 - 13} Currently, simple declaration of all relationships with industry is considered sufficient, although to be meaningful, disclosure must be complete. Present and recent past direct monetary relationships must be divulged in detail, along with any independent investment or contractual relationship conferring potential future financial benefit.¹⁴ Moreover, the prominence gained from participation in a guideline process can translate into sizeable speaking fees. Therefore, disclosure cannot stop just with publication of the guidelines because payment in the future can be the reward for actions in the past.

It is not the number of entanglements with industry—it is their size, individually and in toto. Therefore, the sums should be reported. The argument has been made that ethical equipoise can be achieved by accepting rewards from all the competitors in a field.¹⁵ However, accepting financial reward from everyone does not equal being beholden to no one. Indeed, once the sum of the benefits is large enough, no matter how it is partitioned, the presumption that physicians are unaffected is not credible.

The following reforms of the guideline process are suggested. First, the requisite membership of guideline groups should be defined and include the expertise relevant to that discipline plus epidemiologists, statisticians, and experts in health care policy. Second, the largest part of the guideline committee membership, and in particular the leaders, should be changed from one edition to the next and each edition of the guideline should include an expiration date. Third, reports should not be issued unanimously unless all members fully agree to all sections. Alternate interpretations and viewpoints should be recorded and issued along with the majority opinions. Fourth, posting an almost final version on the Internet and inviting commentary is an attractive model. This helps ensure that where legitimate differences of scientific opinion exist, there is an opportunity for exchange before final decisions are taken. Fifth, before publication, guidelines should undergo independent scientific review. The journal editor should present the criticisms and suggestions that result from the reviewers to the panel for its responses, and may require revision of the guideline document, as appropriate. The editor also should consider co-publishing alternate points of view as necessary.

Sixth, all financial relationships with industry should be disclosed in detail, including amounts received, and should be publicly available. Receipt of substantial benefits from any company or series of companies whose products might be under consideration should disqualify that individual from participation in any guideline decision-making process. Potential and actual financial benefits gained by the authors for the next 2 years should also be limited and disclosed to the association that sponsored the guideline process.

Seventh, associations that sponsor and promote guidelines should create joint codes to govern conflict of interest both on the part of participants in the guideline process and the associations. Associations should not accept money from industry to sponsor, underwrite, or promote guidelines.

In summary, evidence is complex and incomplete. Therefore, when the evidence warrants, guidelines should respect diversity of views. Guidelines must be directed only to the interests of patients and not to those who profit from them. Failure to reform the guideline process risks replacing one authority-based system with another, whereas the core objective should be to strengthen an evidence-based approach to improve clinical care.