Accuracy of Positron Emission Tomography for Diagnosis of Pulmonary Nodules and Mass Lesions: Title and subTitle BreakA Meta-analysis FREE

Focal pulmonary lesions are commonly encountered in clinical practice. Such lesions may be classified as nodules or masses, depending on their size. Pulmonary nodules are circumscribed parenchymal lesions that measure less than 3 to 4 cm in diameter and are completely surrounded by aerated lung. Most nodules are asymptomatic. Approximately half prove to be malignant.¹ Pulmonary masses measure more than 3 to 4 cm in diameter, often cause symptoms, and are even more likely to be malignant. Malignant nodules and masses usually represent primary bronchogenic carcinoma, although pulmonary metastases are common in patients with nonthoracic primary neoplasms and in patients with multiple nodules.

Diagnostic evaluation of focal pulmonary lesions should be both accurate and efficient to facilitate prompt resection of malignant tumors when possible. For evaluation of mass lesions, common diagnostic studies include sputum cytology and bronchoscopy when the lesion is centrally located and transthoracic needle biopsy when the mass is located in the lung periphery. Controversy still exists regarding optimal management of solitary nodules.² Computed tomography (CT) is useful for localizing the nodule within the lung parenchyma, and CT densitometry sometimes detects calcification that suggests a benign etiology.³ Other CT findings, such as spiculation, markedly increase likelihood of malignancy.⁴ Management options include surgical resection, transthoracic needle biopsy, and observation with serial chest radiographs. Each approach has advantages and disadvantages. Surgery is the diagnostic criterion standard and the definitive treatment for malignant nodules, but surgery should be avoided in cases of benign disease. Needle biopsy often establishes a specific benign or malignant diagnosis, but biopsy is invasive, potentially risky, and sometimes nondiagnostic. Observation with serial chest radiographs avoids unnecessary surgery in cases of benign disease but delays diagnosis and treatment when malignancy is present.

Recent attention has focused on positron emission tomography (PET) with the glucose analog 18-fluorodeoxyglucose (FDG) as a diagnostic tool for focal pulmonary lesions.^{5 - 7} This noninvasive functional imaging test capitalizes on the observation that malignant cells have increased rates of glucose metabolism⁸ and is gaining acceptance in oncology for tumor diagnosis, disease staging, and evaluation of treatment response.^{9 - 10} When FDG-PET is used for diagnosis of focal pulmonary lesions, proponents argue that observation is safe when results are negative, especially in low-risk populations, because the sensitivity and negative predictive value of FDG-PET are believed to be high.¹¹

Several studies have addressed the diagnostic accuracy of FDG-PET for identifying malignant focal pulmonary lesions, but most studies have enrolled small numbers of participants. Conclusions regarding accuracy are also limited by differences in case mix, especially with respect to lesion size. Study methods have varied in other ways. Some studies have evaluated FDG metabolic imaging without a dedicated PET scanner by using a modified dual-detector gamma camera in coincidence mode.¹² In addition, while most investigators have used qualitative methods for image interpretation, some have attempted to improve diagnostic accuracy using semiquantitative methods.¹³ When qualitative methods are used, a positive test result is often defined as any increase in FDG uptake from background, although more stringent thresholds can be adopted.

We performed a meta-analysis to estimate the diagnostic accuracy of FDG-PET for malignant pulmonary nodules and mass lesions, to assess the methodological quality of FDG-PET studies, and to ascertain if differences in study methods explain variation in study results. In addition, we aimed to determine if the diagnostic accuracy of FDG-PET is similar for pulmonary nodules and larger mass lesions, to evaluate whether semiquantitative interpretation of FDG-PET images improves diagnostic accuracy, and to address whether functional imaging with FDG and a dedicated PET scanner is more accurate than imaging with FDG and a modified gamma camera in coincidence mode.

We used systematic methods to identify relevant studies,¹⁴ assess study eligibility,¹⁵ evaluate study methodological quality,¹⁶ and summarize findings regarding diagnostic accuracy.^{17 - 18}

We attempted to identify all studies that examined functional imaging with FDG for diagnosis of pulmonary nodules and mass lesions. An investigator (M.K.G.) and a professional librarian developed strategies for locating studies published between January 1966 and September 2000 in the MEDLINE and CANCERLIT databases. We augmented our computerized literature search by manually reviewing the reference lists of identified studies, scanning abstracts from recent conference proceedings, and directly contacting investigators. We included studies published in any language.

Two investigators (M.K.G. and W.G.K.) independently evaluated potential English-language studies for inclusion and subsequently resolved disagreements by discussion. One reviewer evaluated non–English-language studies. Reviewers of English-language studies were blinded to journal, author, institutional affiliation, and date of publication. We included studies in any language that (1) examined FDG-PET or FDG with a gamma camera in coincidence mode for diagnosis of pulmonary nodules or mass lesions; (2) enrolled at least 10 participants with pulmonary nodules or masses, including at least 5 participants with malignant lesions; and (3) presented sufficient data to allow calculation of sensitivity and specificity for malignancy. We included abstracts in the main analysis only when study authors provided full reports of their methods and results. When necessary, we attempted to contact authors who published more than 1 study to establish whether they reported results for overlapping patient populations. Forty eligible studies were selected from 727 potentially relevant studies (Table 1).

We designed our eligibility criteria to identify studies that met minimal standards for acceptability. To identify high-quality studies, we adapted criteria for methodological quality proposed by Kent et al,¹⁶ who evaluated imaging tests for diagnosis of lumbar spinal stenosis. These criteria also have been used to assess quality of studies of polymerase chain reaction for diagnosis of human immunodeficiency virus infection.¹⁸ The revised criteria cover 7 dimensions: technical quality of the index test, technical quality of the reference test, independence of test interpretation, description of the study population, cohort assembly, sample size, and unit of data analysis. To develop criteria for the technical quality of FDG-PET, we referred to guidelines published by the Society of Nuclear Medicine¹⁹ and consulted 2 nuclear medicine physicians experienced in FDG-PET imaging. Two investigators (M.K.G. and W.G.K.) independently assessed English-language studies for methodological quality and subsequently resolved disagreements by discussion. One reviewer assessed the quality of non–English-language studies.

One investigator (M.K.G.) abstracted study data regarding demographic characteristics of participants, size distribution and number of pulmonary nodules or mass lesions, and study inclusion and exclusion criteria. Two investigators (M.K.G. and W.G.K.) independently abstracted data regarding prevalence of malignancy and sensitivity and specificity of the imaging test for malignancy. When possible, we separately tabulated test performance for pulmonary nodules measuring less than 3 to 4 cm in diameter. We also separately tabulated data from studies that used semiquantitative methods for interpreting FDG-PET images and from studies that evaluated FDG imaging with a gamma camera in coincidence mode.

To evaluate agreement between raters for the assessments of study eligibility and methodological quality, we calculated the observed percentage agreement and the κ coefficient for interrater reliability.²⁰

For each study, we constructed 2 × 2 contingency tables in which all participants were classified as having positive or negative FDG-PET results and as having a malignant or benign pulmonary nodule or mass lesion. We calculated the true-positive rate (TPR; sensitivity), the false-positive rate (FPR; 1 − specificity), and the log odds ratio (OR; log odds of TPR − log odds of FPR). The log OR is a measure of diagnostic test performance that accounts for the fact that sensitivity and specificity are positively correlated. To calculate the log OR, we added 0.5 to each cell in any 2 × 2 table that contained 1 or more zero values. We calculated 95% confidence intervals (CIs) for the TPR and the FPR based on normal or Poisson approximations to the binomial distribution, as appropriate.²¹

To quantitatively summarize study results, we used a meta-analytic method to construct summary receiver operating characteristic (ROC) curves.^{17 - 18} Receiver operating characteristic curves illustrate the trade-off between sensitivity and specificity as the threshold for defining a positive test varies from most stringent to least stringent. Our method assumes that each individual study represents a unique point on a common ROC curve. We defined the maximum joint sensitivity and specificity as the point on a symmetrical ROC curve that is intersected by a diagonal line that runs from the top left corner to the bottom right corner of the ROC diagram. This point, at which sensitivity and specificity are equal, is a global measure of test accuracy, similar to the area under the ROC curve. The maximum joint sensitivity and specificity of a perfect test is 1.0, and the maximum joint sensitivity and specificity of a test that has no diagnostic value is 0.5.

Our method for constructing summary ROC curves has been described in detail elsewhere.^{18 ,22} Briefly, we logistically transformed the TPR and FPR and fit a summary ROC curve with linear regression by using the log OR as the dependent variable and an implied function of the test threshold (log odds of TPR + log odds of FPR) as the independent variable.¹⁷ A limitation of this method is that the transformation requires use of a correction factor when the 2 × 2 table for a study contains 1 or more zero values, ie, when sensitivity, specificity, or both are 100%. An advantage of the method is that it provides a statistical test of the hypothesis that the variance in the group with malignant disease and the variance in the group without malignant disease are equal. The variances are not equal when the slope of the regression line is significantly different from zero. When the slope is not different from zero, the resulting ROC curve is symmetrical and can be described by a common or summary log OR. When this condition was met, we used the Mantel-Haenszel method for pooling ORs because this method does not require use of a correction factor.²⁰ The regression method and the Mantel-Haenszel method produced nearly identical results. We calculated the maximum joint sensitivity and specificity by using the formula Q* = (1 + e^−A/2)⁻¹, where Q* is the maximum joint sensitivity and specificity and A is the summary log OR.¹⁷

To determine if the diagnostic accuracy of FDG-PET has improved over time, we ordered studies chronologically and performed a cumulative meta-analysis.²³ To determine if study quality affected diagnostic accuracy, we compared studies that did and did not satisfy each study quality criterion. We also compared studies that satisfied at least 70% of the quality criteria with studies that did not. To evaluate whether semiquantitative interpretation of FDG-PET images improves accuracy, we compared studies that used qualitative and semiquantitative methods of interpretation. To investigate the influence of excluding abstracts for which we could not obtain full reports, we included data from these studies in a secondary analysis. To analyze publication bias, we created inverted funnel plots of individual study log ORs plotted against sample size.²⁴ (An asymmetrical funnel plot would suggest that additional small studies may have been conducted but not published because of unfavorable results.)

To make statistical comparisons between groups of studies, we compared log ORs by using unpaired t tests or the Mann-Whitney U test, as appropriate.

Our search identified 727 potentially relevant studies, including 12 abstracts published since 1998. None of the authors of the abstracts supplied full reports of their methods and results, but 5 abstracts provided sufficient data to permit construction of a 2 × 2 contingency table for diagnostic test performance. We included these results in a sensitivity analysis. We excluded 660 published studies after scanning their titles and abstracts, including 217 studies that evaluated other functional imaging tests, 121 review articles and letters to the editor, 63 studies that examined FDG-PET for applications outside oncology, 56 studies that examined FDG-PET for oncologic applications other than lung cancer, 38 studies that evaluated FDG-PET for lung cancer staging, and 165 studies for miscellaneous reasons. Fifty-five potentially eligible studies were subsequently appraised. Of these, we excluded 15 studies because the study did not address the diagnosis of pulmonary nodules or masses,^{25 - 27} did not enroll the required number of participants,^{28 - 33} did not present sufficient data to permit calculation of sensitivity and specificity,^{28 - 29 ,33 - 38} or presented data that were reported previously.³⁹ Interrater agreement for study eligibility was 94% (κ = 0.82), indicating excellent agreement.

Participant characteristics and study inclusion and exclusion criteria in the 40 eligible studies are outlined in Table 1.^{1 - 13 ,40 - 76} The mean age of participants ranged from 55.5 to 70.8 years, and 39% to 99% of participants were men. Most studies enrolled participants who had either pulmonary nodules or mass lesions, but 6 studies limited enrollment to those with pulmonary nodules.^{42 ,50 ,54 ,57 ,60 - 61} Seven other studies provided separate results for participants with pulmonary nodules.^{13 ,40 - 41 ,45 - 47 ,58} The median prevalence of malignancy was 72.5% (interquartile range, 65%-82.8%). Six studies did not enroll any participants with benign nodules or masses and, therefore, reported sensitivity but not specificity.^{68 - 73} In these studies, results should be interpreted with caution because such studies may use a test threshold that maximizes sensitivity without sacrificing specificity. Therefore, we did not include these studies in our meta-analytic estimates of diagnostic performance. Three studies examined FDG imaging with a modified gamma camera^{74 - 76} and 3 other studies examined both FDG-PET and FDG imaging with a modified gamma camera.^{12 ,72 - 73} Five studies reported the accuracy of FDG-PET by using both semiquantitative and qualitative methods of image interpretation.^{44 ,51 ,58 ,61 ,72} Five other studies used semiquantitative methods of interpretation but did not use qualitative methods.^{13 ,46 - 47 ,52 ,59}

The mean interrater agreement for assessment of study quality was 88% (mean κ = 0.67), indicating good agreement.

Because our criteria for study quality were rigorous, no study met all of them (Table 2). Fourteen studies satisfied 70% to 80% of the criteria,^{13 ,41 - 47 ,50 ,54 ,57 - 58 ,61 ,64} 18 studies satisfied 50% to 69% of the criteria,^{12 ,40 ,49 ,53 ,55 - 56 ,59 - 60 ,62 - 63 ,65 - 70 ,72 - 73} and 5 studies met less than 50% of the criteria.^{11 ,48 ,51 - 52 ,71} In general, studies adequately described their technical methods and followed most of the guidelines published by the Society of Nuclear Medicine for performing FDG-PET imaging. However, only 6 studies (16%) indicated that participants with hyperglycemia were excluded. Twenty-four studies (65%) used a dose of FDG that was at least 370 MBq (10 mCi), 17 studies (52%) described an image acquisition time of 6 to 15 minutes, and 23 studies (62%) clearly specified criteria for defining a positive test result.

Most studies used acceptable reference tests, although 2 studies did not require histologic proof of malignancy in all cases,^{11 ,52} and 5 studies did not require histology or 2-year follow-up to establish a benign diagnosis.^{11 ,40 ,48 ,52 ,65} Readers of FDG-PET images were blinded to the final diagnosis in 19 studies (51%), and readers were blinded to clinical and radiological data in 9 studies (24%). In 62% to 78% of studies, authors adequately described the clinical characteristics of the study population by reporting age, sex, and lesion size distributions. Twenty studies (54%) collected data prospectively and 23 studies (62%) enrolled participants who had an undiagnosed focal pulmonary lesion at the time of FDG-PET imaging. Only 13 studies (35%) enrolled more than 35 participants with malignant lesions and only 1 study (3%) enrolled more than 35 participants with benign lesions. In 32 studies (86%), the unit of data analysis was the individual patient, while in the 5 remaining FDG-PET studies, data were presented for more than 1 lesion per patient in at least 1 patient.

For 1474 focal pulmonary lesions of any size, the sensitivity of FDG-PET for detecting malignancy ranged from 83% to 100% (Figure 1). Specificity was extremely variable. The mean sensitivity and specificity were 96.0% and 73.5%, and the median sensitivity and specificity were 97.0% and 77.8%, respectively. For 450 pulmonary nodules, the mean sensitivity and specificity of FDG-PET for detecting malignancy were 93.9% and 85.8%, and the median sensitivity and specificity were 98.0% and 83.3%, respectively. We identified only 8 instances in which results were specified for nodules measuring less than 1 cm in diameter, including 3 true-positive cases, 2 true-negative cases, and 3 false-negative cases.

Figure 2 shows meta-analysis results expressed in terms of summary log ORs. For lesions of any size, the summary log OR for FDG-PET was 4.68 (95% CI, 4.21-5.14), corresponding to a maximum joint sensitivity and specificity of 91.2% (95% CI, 89.1%-92.9%). For pulmonary nodules, the summary log OR for FDG-PET was 4.40 (95% CI, 3.70-5.09), corresponding to a maximum joint sensitivity and specificity of 90.0% (95% CI, 86.4%-92.7%). We found no difference in the accuracy of FDG-PET for pulmonary nodules compared with pulmonary lesions of any size (P = .43). Diagnostic accuracy appeared to be similar for studies that used semiquantitative methods of image interpretation and studies that used qualitative methods (P = .52). Accuracy appeared to be similar for studies of FDG-PET and studies of FDG imaging with a modified gamma camera (P = .19).

Figure 3A shows the summary ROC curve and 95% CIs for FDG-PET for lesions of any size. Most studies used diagnostic thresholds that favored sensitivity rather than specificity. Sensitivity was 96.8% (95% CI, 95.0%-98.0%) at the point on the ROC curve that corresponded to the median specificity of 77.8%. Figure 3B shows the summary ROC curve and 95% CIs for FDG-PET for pulmonary nodules. Sensitivity was 94.2% (95% CI, 89.1%-97.0%) at the point on the summary ROC curve that corresponded to the median specificity of 83.3%.

The cumulative meta-analysis showed that the summary log OR gradually improved from 3.81 (95% CI, 1.24-6.37) in 1990 to 5.09 (95% CI, 4.40-5.78) in 1997, after which time it fell slightly to 4.68 (95% CI, 4.21-5.14). Diagnostic accuracy was better in the 14 studies that met at least 70% of the criteria for methodological quality compared with the remaining studies (P = .007). The only aspect of study quality that had a statistically significant effect on diagnostic accuracy was blinding of FDG-PET image readers to the final diagnosis. The mean log OR was 4.90 (95% CI, 4.15-5.66) for studies that specified that FDG-PET readers were blinded compared with 3.67 (95% CI, 2.92-4.42) for studies that did not specify blinding (P = .006). The summary log OR for data from abstracts was lower than the summary log OR for published articles, but the difference was not statistically significant (P = .27). An inverted funnel plot did not suggest evidence of publication bias.

Positron emission tomography using 18-fluorodeoxyglucose is an accurate test for identifying malignant pulmonary lesions, although little information exists about FDG-PET performance for nodules measuring less than 1 cm in diameter. Because the spatial resolution of the current generation of PET scanners is 7 to 8 mm, use of FDG-PET for smaller nodules should await further technological refinements. Performance of FDG-PET appears to be similar for nodules measuring at least 1 cm in diameter and larger mass lesions. The maximum joint sensitivity and specificity of FDG-PET was 91.2% for pulmonary lesions of any size and 90.0% for pulmonary nodules. The maximum joint sensitivity and specificity is a global measure of test performance and does not represent the actual or optimal operating point on the summary ROC curve. In fact, most studies of FDG-PET operated at a test threshold that favored sensitivity rather than specificity, which minimized the number of false-negative test results. Such a threshold is appropriate because the consequences of a false-negative test result (delayed detection of malignancy and possible missed opportunity for surgical cure) are more undesirable than the consequences of a false-positive test result (unnecessary biopsy or surgery). In current practice, FDG-PET operates at a point on the ROC curve at which sensitivity and specificity for malignancy are approximately 96.8% and 77.8%, respectively. The corresponding likelihood ratios for positive and negative test results are 4.36 and 0.04, respectively.

Our study addressed 2 important clinical issues related to metabolic imaging with FDG. First, we failed to show that semiquantitative image interpretation improves the accuracy of FDG-PET. Second, we found no evidence that FDG-PET is more accurate than metabolic imaging with FDG and a modified gamma camera. However, our estimates for FDG imaging with a modified gamma camera are based on a few small studies, and the 95% CI for the summary log OR is wide. Additional research is needed to clarify this important question.

Sensitivity analysis showed that adding data from recent abstracts did not substantially change the results. The only aspect of study design that affected the accuracy of FDG-PET was blinding of image readers to the final diagnosis. Surprisingly, studies that specified blinded interpretation showed that FDG-PET was more accurate than studies that did not specify blinding. The reason for this is unclear, but 1 possibility is that more rigorous blinded studies may have been conducted at FDG-PET centers with more technical experience. This is supported by the observation that diagnostic accuracy was better for higher-quality studies. The cumulative meta-analysis showed that diagnostic accuracy gradually improved from 1990 to 1997, then declined slightly. The initial improvement might be explained by advances in FDG-PET technology or by increasing experience of FDG-PET image readers. The subsequent decline could be the result of enrolling less highly selected patient populations or dissemination of FDG-PET to centers with less experience.

This study has several limitations. First, our meta-analytic method produces a summary ROC curve (which best characterizes diagnostic test performance), but it does not precisely identify the actual or optimal operating point on the curve. To determine point estimates for sensitivity and specificity, we identified the point on the summary ROC curve that corresponded to the median specificity. Other approaches are possible but not necessarily better. Second, our estimates of diagnostic accuracy do not capture all of the potential benefits of FDG-PET imaging in patients with pulmonary nodules and mass lesions. Positron emission tomography with 18-fluorodeoxyglucose is more accurate than CT for detecting regional lymph node metastases,⁷⁷ which may occur in up to 21% of patients with stage T1 lung cancer,^{78 - 79} and FDG-PET may also detect distant metastases.³⁵ Even false-positive findings of FDG-PET may have clinical value because such findings may alert clinicians to the presence of active inflammatory or infectious processes that require further evaluation.

Is FDG-PET potentially helpful in clinical decision making? If we assume that the pretest probability of malignancy is 20% for a hypothetical low-risk patient with a pulmonary nodule, the posttest probability is about 1% when the FDG-PET result is negative. In contrast, for a high-risk patient (eg, pretest probability = 80%) with negative findings on FDG-PET, the posttest probability of malignancy is 14%. Thus, the negative predictive value depends on the pretest probability of disease. Several prediction rules exist for determining the pretest probability of malignancy.^{4 ,80 - 81} For low-risk patients, FDG-PET has a high negative predictive value and observation is probably safe when FDG-PET shows no uptake.

Policy-level decisions regarding dissemination of FDG-PET must consider not only diagnostic accuracy but also clinical outcomes and costs. At present, Medicare reimbursement for FDG-PET imaging is approximately $1912.⁸² In comparison, reimbursement for noncontrast CT of the thorax is $276 and reimbursement for CT-guided needle biopsy is approximately $560.⁸³ Formal cost-effectiveness studies are needed to determine if diagnostic strategies that include FDG-PET represent a good value for the health care dollar.