Diagnostic Performance of Computed Tomography Angiography in Peripheral Arterial Disease: Title and subTitle BreakA Systematic Review and Meta-analysis FREE

Imaging is necessary for planning interventions in patients with lower extremity peripheral arterial disease (PAD). Quiz Ref IDNoninvasive imaging modalities, including duplex ultrasonography, magnetic resonance angiography (MRA), and computed tomography angiography (CTA) are available for grading lower extremity arterial disease. Duplex ultrasonography has a high specificity of 95% and a somewhat lower sensitivity of 88% for detecting hemodynamically significant lesions (>50% stenosis or occlusion).¹ Gadolinium-enhanced MRA appears to be more accurate than duplex ultrasonography, with a specificity of 96% (range, 91%-99%) and a sensitivity of 98% (range, 92%-100%).^{1 - 3}

Computed tomography angiography is increasingly attractive due to rapid technical developments. Shorter acquisition times, thinner slices, higher spatial resolution, and improvement of multidetector computed tomographic (CT) scanners enable scanning of the whole vascular tree in a limited period with a decreasing (but still substantial) amount of contrast medium. Recent studies on CTA report sensitivity and specificity rates of around 98% for detecting PAD.^{4 - 5} Figure 1 illustrates an occluded left common iliac artery detected with CTA and the same lesion visualized with intra-arterial digital subtraction angiography (DSA).

Apart from differentiating between the presence vs absence of a significant atherosclerotic lesion in a vessel, the distinction between a significant stenosis and occlusion is important and guides invasive management.⁶ Traditionally, systematic reviews of diagnostic test accuracy use dichotomized data to calculate summary estimates of test performance. However, many diagnostic studies present their outcomes not as dichotomous outcomes, but according to a scale of disease severity (eg, percent arterial stenosis). The purpose of this systematic review and meta-analysis was to determine the diagnostic performance of CTA compared with intra-arterial DSA for grading disease severity in patients with PAD.

MEDLINE (January 1966-August 2008), EMBASE (January 1980-August 2008), and the Database of Abstracts of Reviews of Effectiveness were searched for relevant publications with the assistance of a clinical librarian. We used Medical Subject Headings (MeSH) terms and accompanying entry terms for the patient group (patients with peripheral arterial occlusive disease), the diagnostic test (CTA), the reference standard (conventional angiography or DSA), and outcome (sensitivity and specificity).

Details of the search strategy are shown in Table 1. The reference lists of all papers selected for assessment of full-text and of all retrieved systematic and narrative reviews were reviewed for search completion. There were no language restrictions. Titles and abstracts were screened by 2 reviewers (R.M. and M.J.W.K.) to identify potentially relevant articles. Discrepancies in judgment were resolved after discussion. Studies meeting inclusion criteria were those comparing multidetector CTA with intra-arterial catheter angiography or intra-arterial DSA as the reference standard that included at least 10 patients with intermittent claudication or critical limb ischemia. Inclusion criteria also required an outcome measure of more than 50% stenosis or arterial occlusion and presentation of either 2 × 2 or 3 × 3 contingency tables or data allowing their construction. Studies on CTA performed in follow-up after lower extremity revascularization and duplicate publications were excluded.

Methodological quality of included studies was assessed independently by 2 observers (R.M. and M.J.W.K.) using the QUADAS tool, which is a quality assessment tool specifically developed for systematic reviews of diagnostic accuracy studies.⁷ In using this tool,⁷ we defined a representative patient spectrum as a cohort that included both participants with claudication and those with critical limb ischemia. The optimal time interval between CTA and the reference standard was defined as 30 days or less. The QUADAS components also included clarity of the methodological description for CTA and the reference standard and whether interpretation of CTA and the reference standard were blinded. We also recorded whether clinical information was available during the diagnostic test interpretation, because this information is available when the diagnostic test is used in clinical practice.

Data extraction was performed independently by 2 reviewers (R.M. and M.J.W.K.). Discrepancies were resolved by discussion. Data extraction included characteristics of the study population, methodologic details for the CTA and reference standard, and outcome data. Many studies subdivided an artery into multiple segments. A segment with more than 50% stenosis or an occlusion was considered diseased. A segment with 50% or less stenosis was considered nondiseased. We constructed 2 × 2 contingency tables for the entire vascular tree from the abdominal aorta through the ankles. If data were available, tables were constructed by anatomic region (the aortoiliac, femoropopliteal, and infrapopliteal regions).

For optimal planning before invasive treatment for PAD, it is essential to have information on both the presence and extent of disease. It is important to be able to differentiate between a stenosis and occlusion. Therefore, if data were available, 3 × 3 contingency tables were constructed for each arterial segment. Segments were graded according to 3 categories: 1 (no stenosis or ≤50% stenosis), 2 (>50% stenosis but not occluded), and 3 (occlusion).

When data were provided separately for 2 observers, we used raw data to calculate the mean of both data sets. When outcome data were provided for different patient groups, numbers of true-positive, false-negative, false-positive, and true-negative results were combined before analysis. Discrepancies were resolved by discussion.

Sensitivity and specificity summary estimates for detecting more than 50% stenosis or an occlusion were calculated on a per-segment basis for the entire vascular tree from the abdominal aorta through the ankles using bivariate models with either random-effects or fixed-effects approaches,⁸ depending on the presence of statistical heterogeneity. Statistical heterogeneity was defined as an I² statistic value of more than 50%.⁹ The bivariate model implies that when data of one accuracy parameter show statistical heterogeneity, both accuracy parameters will be calculated in a random-effects model. When statistical heterogeneity was identified, we attempted to identify its source by performing subgroup analyses. These subgroups are based on the most likely potential sources of bias (ie, patients studied [intermittent claudication vs critical limb ischemia], study design [prospective vs retrospective], and CTA execution [number of multidetector CT slices: 16- or 64-slice vs 2- or 4-slice]).¹⁰ Outcome comparisons for sensitivity and specificity between subgroups were performed by using z tests.

The influence of study quality on outcome was assessed by using the QUADAS tool. For each study, a score was calculated for each QUADAS component as follows. A score of +2 was assigned for each QUADAS criterion that was met, a score of 0 was assigned if the QUADAS criterion was not met, and a score of +1 was assigned if it was unclear whether the criterion was met. Points were summed to achieve a final score for each study. Two subgroups of studies were defined according to whether their total quality score was less than or equal to the median quality score vs more than the median quality score. Outcomes for sensitivity and specificity were compared between these subgroups using z tests. These analyses were repeated using a distinct threshold for quality (ie, <median score for quality vs ≥median score for quality).

If data were available, summary estimates for sensitivity and specificity on a per-segment basis for detection of either more than 50% stenosis or occlusion by anatomic region (aortoiliac, femoropopliteal, and tibial arteries) were calculated by means of bivariate models using either random-effects approaches or fixed-effects approaches, depending on the I² value. If data were available, summary estimates for the 3 × 3 contingency tables on a per-segment basis were obtained using a multivariate approach previously described by Bipat et al,¹¹ using either a random-effects approach or a fixed-effects approach, depending on the I² value.

Publication bias was examined by construction of a funnel plot. The x-axis consisted of the natural logarithm of the diagnostic odds ratio, which is the ratio of the multiplication of true-positive and true-negative results and the multiplication of the false-negative and false-positive results. On the y-axis, we plotted the sample size (ie, the number of patients in the study). Egger's regression test was used to examine the asymmetry of the funnel plot.¹²

The analyses for accuracy parameters were performed in the WinBUGS program, which uses a Bayesian algorithm.¹³ The analysis for publication bias was performed using SPSS version 15.0 (SPSS Inc, Chicago, Illinois). Two-sided P<.05 was considered to be statistically significant.

The initial search yielded 909 articles (Figure 2), including 3 systematic reviews.^{14 - 16} The Database of Abstracts of Reviews of Effectiveness yielded no additional studies. A total of 868 articles were excluded based on review of article titles, abstracts, or both. Interobserver agreement on study selection was excellent, with agreement in 896 of 909 titles (κ statistic = 0.85). The most frequent reason for exclusion was absence of CTA as the index test. Of the 41 full-text publications, 21 were excluded. Twelve publications presented the data in a format that precluded construction of 2 × 2 tables, 2 studies evaluated single-slice CTA, 3 studies combined results for different indications together (both aneurysms and obstructive disease), 1 study included fewer than 10 patients, and 3 studies did not clearly define the indications for CTA testing. The remaining 20 articles were included.^{4 - 5 ,17 - 34}

All studies were diagnostic cohort studies. No randomized controlled trials fulfilled the inclusion criteria because randomized controlled trials generally compare 2 different imaging modalities and do not perform both the index test and reference test in the same patient. Our goal in this study was to compare CTA with DSA in the same patients. One study was published in German,³¹ 4 studies were published in Chinese,^{19 ,24 - 25 ,34} and the remaining studies were published in English. The median sample size was 33 (range, 16-279) and these 20 studies included 957 patients. All studies divided the vascular tree into segments, varying from 6 segments to 42 segments (mean, 24 segments). In all studies except 1 study,²⁴ patients were examined bilaterally. The total number of segments per study varied from 163 to 4743 (median, 730 segments per study).

Table 2 shows characteristics of included studies. Slice thickness varied between 0.75 and 5.0 mm (median, 2.0 mm). Various contrast media were used for the CTA. Iomeprol was used in 6 studies,^{5 ,17 ,20 ,22 - 23 ,28} iopromide was used in 4 studies,^{25 ,30 - 31 ,34} and the remaining studies^{4 ,18 - 19 ,21 ,24 ,26 - 27 ,29 ,32 - 33} used other iodine-based contrast media. The iodine concentration varied between 300 and 400 mg/mL. The amount of administered contrast per scan varied between 88 and 170 mL (median, 130 mL). Interpretation of CTA was always based on the axial images. Other image reconstructions used were maximum-intensity projections (n = 17), volume-rendered technique (n = 15), multiplanar reformation (n = 6), curved-planar reformation (n = 4), and virtual endoscopy (n = 1). In most studies, the interpretation of CTA was performed by 2 observers (n = 16) and sometimes by 3 observers (n = 3)^{18 ,20 ,26} or by 1 observer (n = 1).⁵ Most studies included predominantly patients with intermittent claudication (68% of all patients). Nine studies^{5 ,19 - 20 ,22 ,24 - 25 ,27 ,30 ,34} did not describe clinical symptoms other than peripheral arterial disease. The mean prevalence of diseased segments (occlusion or >50% stenosis) was 29% (SD, 8%; range, 12%-47%). One study³⁰ defined a significant stenosis as at least 70% stenosis.

Table 3 shows methodological assessment of included studies using the QUADAS checklist. Inclusion criteria were clear in only 4 studies.^{20 ,26 ,29 ,32} In most studies, it was unclear whether the same clinical information typically available in clinical practice was available at the time of CTA interpretation. In 7 studies,^{4 - 5 ,19 ,30 ,32 - 34} no clinical information was provided at the time of CTA interpretation. Eleven studies were prospective,^{5 ,17 ,21 - 22 ,25 - 27 ,30 ,32 - 34} 7 retrospective,^{18 - 19 ,23 - 24 ,28 - 29 ,31} and in 2 studies,^{4 ,20} study design methods were unclear. In each study, intra-arterial DSA was used as the reference standard.

Reproducibility. Interobserver agreement for CTA was not provided in 12 studies.^{19 ,21 - 28 ,31 - 32 ,34} One study⁵ included only 1 observer. Seven studies expressed interobserver agreement as κ statistic, ranging between 0.61 and 1.00 (good to excellent agreement).^{4 ,17 - 18 ,20 ,29 - 30 ,33}

Sensitivity and Specificity per Segment for Entire Vascular Tree. Most studies provided data for the entire vascular tree, from the abdominal aorta to the tibial arteries, except for 2 studies.^{24 ,32} Li et al²⁴ provided only data from the femoral to tibial region and Schertler et al³² provided data only for the tibial arteries. Table 2 shows true-positive, false-negative, false-positive, and true-negative results for detecting more than 50% stenosis or occlusion per segment. Three studies presented data for 2 observers separately.^{17 ,21 ,33} For these studies, mean values were presented for the 2 × 2 tables. Statistical heterogeneity was present for sensitivity and specificity (I² values were 92% and 95%, respectively). The summary estimates were 95% for sensitivity (95% confidence interval [CI], 92%-97%) and 96% for specificity (95% CI, 93%-97%).

Table 4 shows the subgroup analyses for identifying potential sources of heterogeneity. Computed tomographic scanner slice number was related to outcome. Studies performed on a 16- or 64-slice multidetector CT scan were more accurate than studies performed on a 2- or 4-slice multidetector CT scan. Study design and population were not significantly associated with diagnostic accuracy.

Study Quality. Subgroup analysis for quality is shown in Table 4. Median study quality was 11 points (range, 6-15 points). Accuracy of low-quality studies (n = 12) did not differ significantly from high-quality studies (n = 8), when low quality was defined as 11 points or less and high quality was defined as more than 11 points (P value for sensitivity comparison = .14 and P value for specificity comparison = .28). Similarly, accuracy of low-quality studies (n = 7) did not differ significantly from high-quality studies (n = 13), when low quality was defined as 10 points or less and high quality was defined as more than 10 points (P value for sensitivity comparison = .81 and P value for specificity comparison = .82).

Sensitivity and Specificity per Segment for Each Anatomic Region. Table 5 shows the true-positive, false-negative, false-positive, and true-negative results and corresponding sensitivity and specificity values for detecting more than 50% stenosis or occlusion according to anatomical region. Summary estimates are provided for different parts of the vascular tree (aortoiliac, femoropopliteal, and tibial arteries). Outcome data by anatomic region were provided in 7 studies.^{4 - 5 ,24 ,27 ,30 ,32 - 33} The summary estimates of sensitivity and specificity for aortoiliac disease provided in 5 studies were 96% (95% CI, 91%-99%; fixed-effects model, I² = 0%) and 98% (95% CI, 95%-99%; fixed-effects model, I² = 29%), respectively. The summary estimates of sensitivity and specificity for femoropopliteal disease provided in 5 studies were 97% (95% CI, 95%-99%; random-effects model, I² = 26%) and 94% (95% CI, 85%-99%; random-effects model, I² = 89%), respectively. The summary estimates of sensitivity and specificity for distal runoff in the tibial arteries provided in 6 studies were 95% (95% CI, 85%-99%; random-effects model, I² = 82%) and 91% (95% CI, 79%-97%; random-effects model, I² = 94%). One study²⁴ that provided results for the femoral artery up to and including the tibial artery reported a sensitivity of 98% and a specificity of 96%.

Correct Diagnosis, Understaging, and Overstaging. Results of the multivariate approach are shown in Table 6. Thirteen studies^{5 ,17 - 22 ,25 - 26 ,28 - 29 ,31 ,34} provided data that allowed construction of 3 × 3 tables. Absolute numbers of segments per category are shown by study. The summary estimates, calculated using a random-effects model, represent the proportions of correct diagnosis, understaging, and overstaging. Computed tomography angiography correctly diagnosed occlusion in 94% of segments. Underestimation of occlusion occurred in 6% of segments, mostly as more than 50% stenosis (5%). Computed tomography angiography correctly diagnosed more than 50% stenoses in 87% of segments. Quiz Ref IDUnderstaging occurred in 9% of segments and overstaging (a significant stenosis was diagnosed by CTA as an occlusion) in 4% of segments. Computed tomography angiography correctly designated a segment without a significant stenosis in 96% of segments and CTA incorrectly designated a segment as occluded when it was free of significant stenosis on DSA in 1 of 1000 cases (0.1%).

Publication Bias. Egger's regression test showed an asymmetric distribution of the points in the funnel plot for detection of publication bias (intercept, 1.67; 90% CI, 0.35-2.99; P = .05), indicating that publication bias was likely (data not shown).

Quiz Ref IDCompared with intra-arterial DSA, our meta-analysis suggests that CTA is highly accurate for assessment of PAD in all regions of the lower extremity arteries. Computed tomography angiography both correctly identified hemodynamically significant lesions and also accurately distinguished between more than 50% stenoses and occlusions. Ninety-four percent of occlusions and 87% of nonoccluded segments with more than 50% stenosis detected by DSA were correctly identified by CTA. The accuracy of CTA may be even higher than reported herein, because all studies used DSA as the reference standard, but did not report whether biplanar views were used to grade disease angiographically. Because CT reconstructions allow 3-dimensional assessment, significant disease may be detected by CTA but be unrecognized by DSA, thus leading to false-positive CTA results and an underestimated specificity.

Our review has several limitations. We excluded 12 studies because they did not provide data allowing construction of 2 × 2 tables. We did not contact these authors to obtain the data, potentially resulting in biased results and less precise estimates of pooled diagnostic accuracy. Our analysis underscores the importance of reporting according to standards such as the STARD to facilitate future meta-analyses.³⁵

One limitation of our study is that we may have missed important publications, despite an extensive search without language restriction.¹² Egger's regression test showed an asymmetric distribution of points in the funnel plot, indicating that publication bias was likely. We used the QUADAS tool for assessing methodological quality of individual studies. This tool was specifically developed for quality assessment of diagnostic accuracy studies included in systematic reviews. The QUADAS tool helped identify severe methodological shortcomings. The QUADAS instrument was used to perform sensitivity analyses according to different levels of quality. We found no relationship between study quality and outcome. A diagnostic test must always be evaluated in a clinically relevant population, because test performance often varies across population subgroups. Nonconsecutive recruitment and inclusion of only a subset of stages of disease can result in spectrum bias. The former may be applicable to the studies included in this review, because only 4 studies clearly described the selection criteria. Many studies provided a scanty description of their patient population, making it difficult to determine whether study participants were representative of PAD patients undergoing CTA in clinical practice. Furthermore, most studies included predominantly patients with intermittent claudication. Such patients are generally treated conservatively and do not typically require a CTA. Patients with critical limb ischemia who require a complete assessment of their lower extremity arteries for planning an open or endovascular intervention were scarce in the studies included in this review. Only 1 study⁴ included a large proportion of patients with critical limb ischemia. Although the authors found that CTA is accurate, more research is needed to determine the clinical value of CTA in the appropriate target population. Finally, in nearly all patients both lower extremities, including asymptomatic legs, were examined. This is reflected by the low prevalence (29%) of diseased segments. It is possible that the diagnostic performance of CTA differs for symptomatic and asymptomatic PAD. Information on the performance of CTA in the symptomatic leg is most relevant for clinical practice.

Another limitation is the data-analysis method in the individual studies. All studies in our meta-analysis divided the vascular tree into segments. The number of segments per patients varied from 6 to 42. The relatively high proportion of segments without a significant stenosis—segments that are likely to be correctly identified by CTA—will result in an overestimation of specificity. From a clinical standpoint, it is more useful to divide the vascular tree into clinically relevant segments (eg, aortoiliac, femoropopliteal, and distal runoff). Finally, the statistical power of this meta-analysis was limited by the relatively small sample size of most included studies. Taking into account all of these methodological issues, results must be interpreted with caution.

Meta-analyses of diagnostic accuracy are used to produce summary estimates of sensitivity and specificity by dichotomization of test results. However, for treatment of patients with PAD, it is important to know both whether a vessel is diseased and also the extent of disease. Staging often consists of 3 or more categories (eg, a normal vessel, a significant stenosis [>50%], or an occlusion). When data are dichotomized, staging information is lost. Using previously described methods,¹¹ we presented 3 × 3 contingency tables along with summary estimates. Computed tomography angiography performed well in detecting occlusions (94% of occluded segments were correctly diagnosed by CTA). Most of the occlusions missed by CTA were diagnosed as more than 50% stenosis. However, for detecting a nonocclusive stenosis of more than 50%, CTA performed less well (CTA underestimated the degree of stenosis in 9% of nonoccluded segments with >50% stenosis).

Overstaging may easily occur when a calcified patent vessel is mistaken for an occluded vessel due to the blooming effect of the calcium.³⁶ However, overstaging occurred less frequently in comparison with understaging. A reason for understaging, given by Martin et al,²⁶ is that DSA may misclassify patent segments as occluded. Possible reasons for this phenomenon are motion artifacts, different rates of calf vessel filling, or insufficient arterial opacification distal to occlusions.²⁶ Perhaps smaller slice thickness will lead to less partial volume effect and hopefully to more accurate measurement of degree of stenosis.

The diagnostic accuracy of CTA seems to compare well with MRA,^{2 - 3} although studies directly comparing these imaging modalities are lacking. Choosing between different imaging modalities for PAD requires consideration of factors in addition to diagnostic accuracy. Duplex ultrasonography has a lower sensitivity than MRA and CTA but it is easily accessible and does not require radiation or contrast agents.¹ However, a recent study showed that duplex ultrasonography was less clinically useful because the therapeutic confidence of the treating physicians for duplex ultrasonography was lower compared with MRA or CTA.³⁷ Magnetic resonance angiography has some important advantages. It does not require iodinated contrast and there is no radiation exposure. Quiz Ref IDHowever, diagnostic costs of MRA are higher than for CTA and there are many contraindications for magnetic resonance imaging, including a pacemaker, metal implants, claustrophobia, and the need for gadolinium contrast agents. These contrast agents can lead to potentially life-threatening nephrogenic systemic fibrosis in patients with renal failure.³⁸ For CTA, iodinated contrast agents are used. These are potentially nephrotoxic, but this adverse effect can be reduced by hydration³⁹ and, although still under debate, administration of acetylcysteine.^{40 - 41}

Another drawback may be that frequent use of multidetector CT scan exposes a patient to large quantities of potentially carcinogenic ionizing radiation.⁴² For example, the standard protocol used in the study by Fraioli et al²² gives a radiation exposure of 13.7 milliSievert (mSv). In the study by Catalano et al,²⁰ the radiation dose is 12.0 mSv. In comparison, a posterior-anterior chest radiography gives a radiation dose of 0.01 mSv and annual background radiation is 3.0 mSv.⁴³ Postprocessing of peripheral CTA can include reconstruction of different images, like curved planar reformations, maximum-intensity projections, and volume renderings. Although clinical interpretation will be facilitated, these reconstructions can be time consuming. Moreover, interpretation of CTA can be seriously hampered in the presence of calcifications as discussed earlier.^{28 - 29 ,44} The choice between different imaging techniques depends on local availability and experience and patient characteristics. Therefore, more studies are needed to determine the exact (additional) value of CTA in comparison with other noninvasive diagnostic modalities for PAD, like MRA and duplex ultrasonography. Randomized trials comparing both modalities and their effect on treatment, outcome, and costs are needed. For future research, we stress the importance of using n × n tables for reporting outcomes of diagnostic studies.

In conclusion, our meta-analysis showed that methodological quality of reports on diagnostic research of CTA is moderate, as expressed by the QUADAS tool. Many studies may have included sources of bias, like spectrum bias and selection bias. Quiz Ref IDNonetheless, CTA was a reliable imaging modality with high sensitivity and specificity for differentiating extent of disease in patients with predominantly intermittent claudication compared with intra-arterial DSA. Our meta-analysis also reveals that the diagnostic performance of CTA for patients with critical limb ischemia has been poorly investigated thus far. More rigorous evaluations of CTA in patients with critical limb ischemia are needed.