PET Scans and Technology Assessment: Title and subTitle BreakDéjà Vu?

Few diagnostic technologies deployed during the past 3 decades have had as profound an impact on clinical medicine as computed tomography (CT) and magnetic resonance imaging (MRI). While these technologies have improved case management, they have also become a billion-dollar industry that has had a major impact on rising health care costs. It is concerning, therefore, that these expensive technologies have been widely adopted before being adequately evaluated.^{1 - 2} With the emergence of positron emission tomography (PET) from research laboratories into routine clinical use, physicians are faced once again with a new technology that promises to deliver more but that also costs more. At the current Medicare reimbursement rate of approximately $2000 per PET scan,³ unfettered use of this technology would easily add billions of dollars in spending to an already stressed health care financing system.

Positron emission tomography is a nuclear medicine imaging technique that uses radiopharmaceuticals—typically, a radionuclide-labeled analog of glucose, fluorodeoxyglucose (FDG)—to detect abnormal metabolic activity. Since malignant tumors usually have increased cellular metabolism and, thus, increased glucose metabolism, PET is able to localize malignant tissue.⁴ By providing information on function and metabolism, PET may complement traditional imaging modes such as CT and MRI, which provide information on anatomical structures. In addition to its ability to distinguish benign and malignant processes based on differences in biological activity, PET has the ability to examine the whole body for both primary malignancies and metastatic disease in a single procedure.⁴

Positron emission tomography was developed in the 1970s and whole-body imaging became possible in the mid-1980s. In 1995, Medicare reimbursement for specific diagnostic uses of PET began. Use of FDG-PET for characterization of suspected solitary pulmonary nodules (SPNs) to plan treatment was begun in 1998.⁴ Medicare coverage for FDG-PET was recently broadened after a request was made to the Health Care Financing Administration (HCFA). Several additional indications were added from a submitted list of 22 diseases and disorders.⁵

It is important to consider how to evaluate promising emerging technologies in light of existing technologies. After safety considerations, the basic issue in the evaluation of any new diagnostic technology focuses on the test performance, typically measured as sensitivity and specificity. Although numerous studies have been published on diagnostic technologies, many do not truly evaluate diagnostic performance, provide no useful information, or are of poor methodological quality. To help understand the literature, systematic review has become the standard approach to evaluate and summarize clinical evidence.⁶ As part of the critical evaluation of the evidence, a comprehensive search for and critical appraisal of the studies are performed. Meta-analyses are then performed to provide quantitative results and allow meaningful comparisons of competing technologies.

In this issue of THE JOURNAL, Gould and colleagues⁷ report their meta-analysis of the diagnostic accuracy of FDG-PET for evaluation of pulmonary lesions. The authors identified 34 studies in their systematic review of the literature that were suitable for inclusion in the meta-analysis. None of the evaluated studies met all of the established quality criteria, and only 14 studies satisfied 70% to 80% of the criteria.

The evaluated studies reported a range of sensitivity between 83% and 100% but had specificities ranging from 0% to 100%, which raises doubts about the reliability of this test. The summary estimates of test sensitivity and specificity to detect malignancy in all focal pulmonary lesions were 96.0% and 73.5%, respectively.

Gould and colleagues allude to the important issue of testing thresholds, ie, pretest probability of disease at which the results of the diagnostic test may affect treatment.⁸ In patients with sufficiently low probability of disease, a positive test result may not sufficiently increase the likelihood of disease to warrant treatment or further evaluation (eg, if a positive test result increases the likelihood of cancer from 1% to 2%). But patients with sufficiently high probability of disease should receive treatment regardless of test findings. In patients with a high pretest probability of malignant disease (eg, 80%), the posttest probability of malignancy with a negative PET result is about 14%. In these patients, regardless of PET results, further evaluation would likely be needed. Use of PET should be limited to patients for whom the test will likely alter treatment. However, the question then arises: at what probabilities of malignancy is it beneficial or cost-effective to avoid further diagnostic testing or to treat patients empirically? Cost-effectiveness analysis, based on reliable estimates of diagnostic accuracy, treatment efficacy, and natural history of the condition of interest, can provide useful guidance. Given the need to balance the risks of diagnostic tissue sampling of lung masses with the risks of failure to diagnose cancer, along with the unknown benefit of early diagnosis of lung cancer, the question of which patients to evaluate is clinically very important.

This type of analysis depends on the test accuracy estimated by the meta-analyses. Several reviews of the literature regarding PET have shown a great deal of variability among specific study questions.^{7 ,9 - 10} For example, PET studies in the meta-analysis by Gould et al included evaluation of SPNs found on a chest radiograph, other pulmonary lesions, extrathoracic lesions, and known lung cancer. The request for broadened Medicare coverage of PET argued that all cancers should be approved as a single indication because the physiologic behaviors of different cancers are similar and, therefore, there is no need to evaluate different cancers separately.⁵ However, the HCFA Medicare Coverage Advisory Committee concluded that the performance of PET may differ depending on the specific cancer being evaluated, the physical location of the cancer, and any possible metastases.⁵ To allow meaningful evaluation of PET (and other diagnostic tests), researchers should ensure that their study questions are clear and precise and that the study population, condition of interest, intervention, and outcome are all well-defined.¹¹ For complete evaluation, it is important that sufficient information is provided to allow further analysis of variables of interest, such as sex, age, smoking status, and, in the case of pulmonary masses, mass size.

A diagnostic test should not be evaluated in isolation from other tests, since this approach provides no useful information about how the new technology should be used in conjunction with existing tests. To determine the proper role of the new technology and to appreciate incremental benefits (ie, whether PET should replace CT and MRI or be used in addition to CT and MRI), studies explicitly addressing these questions need to be performed. Unfortunately, for most diagnostic tests there is a paucity of studies that evaluate the incremental improvement in diagnostic accuracy or of clinical outcomes. Currently, PET is generally used after more easily obtainable tests such as CT. In fact, for Medicare coverage of PET, evidence of a primary tumor, usually by CT, is required. The incremental value of PET over tests such as CT, however, is unknown. Even if PET is highly accurate, if it does not substantially improve on less costly tests, it might be of limited value. While this question may be answered in part by cost-effectiveness analyses, the most useful method of determining the incremental value of a test is by direct comparison in a primary research study.

In addition, improved test performance does not necessarily translate into meaningful changes in clinical treatment decisions; nor does altered treatment necessarily translate into improved patient outcomes.¹ Thus, to fully assess a new diagnostic technology, the evaluation must assess test performance as well as the impact of the test on decision making and clinical outcomes, such as morbidity and mortality.

Proponents of new technologies often speculate that their widespread deployment could lead to reduced health care costs. It remains to be determined whether use of PET could avoid unnecessary procedures in some patients and, thus, result in overall net savings. However, this must be proven rather than assumed, and well-designed clinical trials must be conducted to assess this hypothesis.

More than a decade ago, Chalmers¹² raised 2 issues that continue to be relevant. First, he proposed that insurers pay for the cost of the technologies being assessed. It is encouraging to note that HCFA now pays for the use of new technologies for patients enrolled in clinical trials.^{5 ,13} Managed care organizations and the PET industry also should follow this example. Second, Chalmers proposed standards for early evaluation of new technologies. Unfortunately, judging from the poor quality of the studies assessed in the current meta-analysis by Gould and colleagues and in the assessments of PET by others,^{7 ,9 - 10} these standards do not appear to have been closely followed. To be of value, methodologically rigorous studies must be conducted to address the important questions. Given the billions of dollars at stake and the need to ensure that patients will truly benefit from expensive new technologies, the costs of properly performing methodologically sound clinical studies seem minuscule.