Context Bias: Title and subTitle BreakA Problem in Diagnostic Radiology FREE

Thomas K. P. Egglin, MD; Alvan R. Feinstein, MD

[+] Author Affiliations

Reprints: Thomas K. P. Egglin, MD, Department of Diagnostic Radiology, Yale University School of Medicine, 333 Cedar St, PO Box 208042, New Haven, CT 06520-8042.

Topics in Radiology section editors: Ronald G. Evens, MD, Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, Mo; Charles Clayman, MD, Contributing Editor, JAMA.

JAMA. 1996;276(21):1752-1755. doi:10.1001/jama.1996.03540210060035

Text Size: A A A

Published online

Article

References

ABSTRACT

ABSTRACT | REFERENCES

Objective. —To determine whether radiologists' interpretations of images are biased by their context and by prevalence of disease in other recently observed cases.

Methods. —A test set of 24 right pulmonary arteriograms with a 33% prevalence of pulmonary emboli (PE) was assembled and embedded in 2 larger groups of films. Group A contained 16 additional arteriograms, all showing PE involving the right lung, so that total prevalence was 60%. Group B contained 16 additional arteriograms without PE so that total prevalence was 20%. Six radiologists were randomly assigned to see either group first and then "cross over" to review the other group after a hiatus of at least 8 weeks. The direction of changes in a 5-point rating scale for the 2 readings of each film in the test set was compared with the sign test; mean sensitivity, specificity, and areas under receiver operating characteristic (ROC) curves were compared with the paired t test.

Results. —In the context of group A's higher disease prevalence, radiologists shifted more of their diagnoses toward higher suspicion than expected by chance (P=.03, sign test). In group A, mean sensitivity for diagnosing PE was significantly higher (75% vs 60%; P=.04), and area under the ROC curve was significantly larger (0.88 vs 0.82; P=.02)

Conclusions. —Radiologists' diagnoses are significantly influenced by the context of interpretation, even when spectrum and verification bias are avoided. This "context bias" effect is unique to the evaluation of subjectively interpreted tests, and illustrates the difficulty of obtaining unbiased estimates of diagnostic accuracy for both new and existing technologies.

REFERENCES

ABSTRACT | REFERENCES

Begg CB. Biases in the assessment of diagnostic tests . Stat Med . 1987;;6:411-423.

Schreiber MH. The clinical history as a factor in roentgenogram interpretation . JAMA . 1963;;185:137-139.

Chalmers TC. PET scans and technology assessment . JAMA . 1988;;260:2713-2715.

McNeil BJ, Hanley JA, Funkenstein HH, Wallman J. Paired receiver operating characteristic curves and the effect of history on radiographic interpretation . Radiology . 1983;;149:75-77.

Babcook CJ, Norman GR, Coblentz CL. Effect of clinical history on the interpretation of chest radiographs in childhood bronchiolitis . Invest Radiol . 1993;;28:214-217.

Berbaum KS, el Khoury GY, Franken EAJ, Kathol M, Montgomery WJ, Hesson W. Impact of clinical history on fracture detection with radiography . Radiology . 1988;;168:507-511.

Weinstein MC, Fineberg HV. Clinical Decision Analysis . Philadelphia, Pa: WB Saunders Co; 1980;: 75-130.

Feinstein AR. Editorial rumination: the inadequacy of binary models for the clinical reality of three-zone diagnostic decisions . J Clin Epidemiol . 1990;;43:109-113.

Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve . Radiology . 1982;;143:29-36.

Zar JH. Biostatistical Analysis . 2nd ed. Englewood Cliffs, NJ: Prentice-Hall Inc; 1984;.

Lachs MS, Nachamkin I, Edelstein PH, Goldman J, Feinstein AR, Schwartz JS. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection . Ann Intern Med . 1992;;117:135-140.

Ransohoff D, Feinstein A. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests . N Engl J Med . 1978;;299:926-930.

Sox HCJ, Blatt MA, Higgins MC, Marton KI. Medical Decision Making . Boston, Mass: Butterworth-Heinemann; 1988;:103-145.

Egglin TK, Rummeny E, Stark DD, Wittenberg J, Saini S, Ferrucci JT. Hepatic tumors: quantitative tissue characterization with MR imaging . Radiology . 1990;;176:107-110.

Greenes RA, Begg CB. Assessment of diagnostic technologies: methodology for unbiased estimation from samples of selectively verified patients . Invest Radiol . 1985;;20:751-756.

Black W, Armstrong P. Communicating the significance of radiologic test results: the likelihood ratio . AJR Am J Roentgenol . 1986;;147:1313-1318.

Baron JA. Uncertainty in Bayes . Med Decis Making . 1994;;14:46-51.

Brenner H, Gefeller O. Use of the positive predictive value to correct for disease misclassification in epidemiologic studies . Am J Epidemiol . 1993;;138:1007-1015.

Cooper LS, Chalmers TC, McCally M, Berrier J, Sacks HS. The poor quality of early evaluations of magnetic resonance imaging . JAMA . 1988;;259:3277-3280.

Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists' interpretations of mammograms . N Engl J Med . 1994;;331:1493-1499.

Shapiro S, Venet W, Strax P, Venet L, Roeser R. Ten- to fourteen-year effect of screening on breast cancer mortality . J Natl Cancer Inst . 1982;;69:349-355.

Baker LH. Breast cancer detection demonstration project: five-year summary report . CA Cancer J Clin . 1982;;32:194-225.

Kovanlikaya A, Loro ML, Hangartner TN, Reynolds RA, Roe TF, Gilsanz V. Osteopenia in children: CT assessment . Radiology . 1996;;198:781-784.

Sugimoto H, Takeda A, Masuyama J-I, Furuse M. Early-stage rheumatoid arthritis: diagnostic accuracy of MR imaging . Radiology . 1996;;198:185-192.

Kang E-Y, Staples CA, McGuinness G, Primack SL, Müller NL. Detection and differential diagnosis of pulmonary infections and tumors in patients with AIDS: value of chest radiography versus CT . AJR Am J Roentgenol . 1996;;166:15-19.

Shemesh J, Apter S, Rozenman J, et al. Calcification of coronary arteries: detection and quantification with double helix CT . Radiology . 1995;;197:779-783.

Westra SJ, Zaninovic AC, Vargas J, Hall TR, Boechat MI, Busuttil RW. The value of portal vein pulsatility on duplex sonograms as a sign of portal hypertension in children with liver disease . AJR Am J Roentgenol . 1995;;165:167-172.

Castagnone D, Rivolta R, Rescalli S, Baldini MI, Tozzi R, Cantalamessa L. Color Doppler sonography in Graves' disease: value in assessing activity of disease and predicting outcome . AJR Am J Roentgenol . 1996;;166:203-207.

Figures

Tables

Interactive Graphics

Video

Country-Specific Mortality and Growth Failure in Infancy and Yound Children and Association With Material Stature

Use interactive graphics and maps to view and sort country-specific infant and early dhildhood mortality and growth failure data and their association with maternal