0
May 8, 1987, Vol 257, No. 18 >
ARTICLE | May 8, 1987

Are All Significant P Values Created Equal?: Title and subTitle BreakThe Analogy Between Diagnostic Tests and Clinical Research FREE

Warren S. Browner, MD, MPH; Thomas B. Newman, MD, MPH
[+] Author Affiliations

Reprint requests to the Division of General Internal Medicine 111A1, Veterans Administration Medical Center, San Francisco, CA 94121 (Dr Browner).


JAMA. 1987;257(18):2459-2463. doi:10.1001/jama.1987.03390180077027
Text Size: A A A
Published online
Article
References

Just as diagnostic tests are most helpful in light of the clinical presentation, statistical tests are most useful in the context of scientific knowledge. Knowing the specificity and sensitivity of a diagnostic test is necessary, but insufficient: the clinician must also estimate the prior probability of the disease. In the same way, knowing the P value and power, or the confidence interval, for the results of a research study is necessary but insufficient: the reader must estimate the prior probability that the research hypothesis is true. Just as a positive diagnostic test does not mean that a patient has the disease, especially if the clinical picture suggests otherwise, a significant P value does not mean that a research hypothesis is correct, especially if it is inconsistent with current knowledge. Powerful studies are like sensitive tests in that they can be especially useful when the results are negative. Very low P values are like very specific tests; both result in few false-positive results due to chance. This Bayesian approach can clarify much of the confusion surrounding the use and interpretation of statistical tests.

(JAMA 1987;257:2459-2463)

REFERENCES

1 +
Schade DS, Mitchell WJ, Griego G:  Addition of sulfonylurea to insulin treatment in poorly controlled type II diabetes: A double-blind, randomized clinical trial . JAMA 1987;;257:2441-2445.
2 +
Cramer DW, Goldman MB, Schiff I, et al:  The relationship of tubal infertility to barrier method and oral contraceptive use . JAMA 1987;;257: 2446-2450.
3 +
Bennett KJ, Sackett DL, Haynes RB, et al:  A controlled trial of teaching critical appraisal of the clinical literature to medical students . JAMA 1987;;257:2451-2454.
4 +
Chaiken BP, Williams NM, Preblud SR, et al:  The effect of a school entry law on mumps activity in a school district . JAMA 1987;;257:2455-2458.
5 +
Diamond GA, Forrester JS:  Clinical trials and statistical verdicts: Probable grounds for appeal . Ann Intern Med 1983;;98:385-394.
6 +
Nugent CA, Warner HR, Dunn JT, et al:  Probability theory in the diagnosis of Cushing's syndrome . J Clin Endocrinol Metab 1964;;24:621-627.
7 +
Crapo L:  Cushing's syndrome: A review of diagnostic tests . Metabolism 1979;;28:955-977.
8 +
Vecchio TJ:  Predictive value of a single diagnostic test in unselected populations . N Engl J Med 1966;;274:1171-1173.
9 +
Boyd JC, Marr JJ:  Decreasing reliability of acid-fast smear techniques for detection of tuberculosis . Ann Intern Med 1975;;82:489-492.
10 +
Jones RB:  Bayes' theorem, the exercise ECG, and coronary artery disease . JAMA 1979;;242: 1067-1068.
11 +
Diamond GA, Forrester JS:  Analysis of probability as an aid in the clinical diagnosis of coronaryartery disease . N Engl J Med 1979;;300:1350-1358.
12 +
Griner PF, Mayewski RJ, Mushlin AI, et al:  Selection and interpretation of diagnostic tests and procedures: Principles and applications . Ann Intern Med 1981;;94:553-600.
13 +
Havey RJ, Krumlovsky F, delGreco F, et al:  Screening for renovascular hypertension: Is renal digital-subtraction angiography the preferred noninvasive test? JAMA 1985;;254:388-393.
14 +
Bayes T:  An essay towards solving a problem in the doctrine of chances . Philos Trans R Soc Lond 1763 ;53:370-418.
15 +
Phillips LD: Bayesian Statistics for Social Scientists . New York, Crowell, 1974;.
16 +
Donner A:  A Bayesian approach to the interpretation of subgroup results in clinical trials . J Chronic Dis 1982;;35:429-435.
17 +
Pater JL, Willan AR:  Clinical trials as diagnostic tests . Controlled Clin Trials 1984;;5:107-113.
18 +
Thomas DC, Siemiatycki J, Dewar R, et al:  The problem of multiple inferences in studies designed to generate hypotheses . Am J Epidemiol 1985;; 122:1080-1095.
19 +
Mushlin AI:  Diagnostic tests in breast cancer: Clinical strategies based on diagnostic probabilities . Ann Intern Med 1985;;103:79-85.
20 +
 Lipid Research Clinics Program: The Lipid Research Clinics Coronary Primary Prevention Trial results: I. Reduction in incidence of coronary heart disease . JAMA 1984;;251:351-364.
21 +
MacMahon B, Yen S, Trichopolous D, et al:  Coffee and cancer of the pancreas . N Engl J Med 1981;;304:630-633.
22 +
Jick H, Dinan BJ:  Coffee and pancreatic cancer . Lancet 1981;;2:92.
23 +
Goldstein HR:  No association found between coffee and cancer of the pancreas . N Engl J Med 1982;;306:997.
24 +
Wynder EL, Hall NEL, Polansky M:  Epidemiology of coffee and pancreatic cancer . Cancer Res 1983;;43:3900-3906.
25 +
Kinlen LJ, McPherson K:  Pancreas cancer and coffee and tea consumption: A case-control study . Br J Cancer 1984;;49:93-96.
26 +
Gold EB, Gordis L, Diener MD, et al:  Diet and other risk factors for cancer of the pancreas . Cancer 1985;;55:460-467.
27 +
Hsieh C, MacMahon B, Yen S, et al:  Coffee and pancreatic cancer (chapter 2) . N Engl J Med 1986;;315:587-589.
28 +
Frieman JA, Chalmers TC, Smith H Jr, et al:  The importance of beta, type II errors and sample size in the randomized control trial: Survey of 71 'negative' trials . N Engl J Med 1978;;299:690-694.
29 +
Young MJ, Bresnitz EA, Strom BL:  Sample size nomograms for interpreting negative clinical studies . Ann Intern Med 1983;;99:248-251.
30 +
Sackett DL:  Bias in analytic research . J Chronic Dis 1979;;32:51-63.
31 +
Winkelstein W Jr:  Smoking and cancer of the uterine cervix: Hypothesis . Am J Epidemiol 1977;;106:257-259.
32 +
Wright NH, Vessey MP, Kenward B, et al:  Neoplasia and dysplasia of the cervix uteri and contraception: A possible protective effect of the diaphragm . Br J Cancer 1978;;38:273-279.
33 +
Harris RWC, Brinton LA, Cowdell RH, et al:  Characteristics of women with dysplasia or carcinoma in situ of the cervix uteri . Br J Cancer 1980;;42:359-369.
34 +
Lyon JL, Gardner JW, West DW, et al:  Smoking and carcinoma in situ of the uterine cervix . Am J Public Health 1983;;73:558-562.
35 +
Godfrey K:  Comparing the means of several groups . N Engl J Med 1985;;313:1450-1456.
36 +
Cupples LA, Heeren T, Schatzkin A, et al:  Multiple testing of hypotheses in comparing two groups . Ann Intern Med 1984;;100:122-129.
37 +
Cole P:  The evolving case-control study . J Chronic Dis 1979;;32:15-27.
38 +
Fleiss JL: Statistical Methods for Rates and Proportions , ed 2. New York, John Wiley & Sons Inc, 1981;, p 14.
39 +
Rothman K:  A show of confidence . N Engl J Med 1978;;299:1362-1363.
40 +
Browner WS, Newman TB:  Confidence intervals . Ann Intern Med 1986;;105:973-974.

Figures

Tables

Interactive Graphics

Video

Country-Specific Mortality and Growth Failure in Infancy and Yound Children and Association With Material Stature

Use interactive graphics and maps to view and sort country-specific infant and early dhildhood mortality and growth failure data and their association with maternal

1 +
Schade DS, Mitchell WJ, Griego G:  Addition of sulfonylurea to insulin treatment in poorly controlled type II diabetes: A double-blind, randomized clinical trial . JAMA 1987;;257:2441-2445.
2 +
Cramer DW, Goldman MB, Schiff I, et al:  The relationship of tubal infertility to barrier method and oral contraceptive use . JAMA 1987;;257: 2446-2450.
3 +
Bennett KJ, Sackett DL, Haynes RB, et al:  A controlled trial of teaching critical appraisal of the clinical literature to medical students . JAMA 1987;;257:2451-2454.
4 +
Chaiken BP, Williams NM, Preblud SR, et al:  The effect of a school entry law on mumps activity in a school district . JAMA 1987;;257:2455-2458.
5 +
Diamond GA, Forrester JS:  Clinical trials and statistical verdicts: Probable grounds for appeal . Ann Intern Med 1983;;98:385-394.
6 +
Nugent CA, Warner HR, Dunn JT, et al:  Probability theory in the diagnosis of Cushing's syndrome . J Clin Endocrinol Metab 1964;;24:621-627.
7 +
Crapo L:  Cushing's syndrome: A review of diagnostic tests . Metabolism 1979;;28:955-977.
8 +
Vecchio TJ:  Predictive value of a single diagnostic test in unselected populations . N Engl J Med 1966;;274:1171-1173.
9 +
Boyd JC, Marr JJ:  Decreasing reliability of acid-fast smear techniques for detection of tuberculosis . Ann Intern Med 1975;;82:489-492.
10 +
Jones RB:  Bayes' theorem, the exercise ECG, and coronary artery disease . JAMA 1979;;242: 1067-1068.
11 +
Diamond GA, Forrester JS:  Analysis of probability as an aid in the clinical diagnosis of coronaryartery disease . N Engl J Med 1979;;300:1350-1358.
12 +
Griner PF, Mayewski RJ, Mushlin AI, et al:  Selection and interpretation of diagnostic tests and procedures: Principles and applications . Ann Intern Med 1981;;94:553-600.
13 +
Havey RJ, Krumlovsky F, delGreco F, et al:  Screening for renovascular hypertension: Is renal digital-subtraction angiography the preferred noninvasive test? JAMA 1985;;254:388-393.
14 +
Bayes T:  An essay towards solving a problem in the doctrine of chances . Philos Trans R Soc Lond 1763 ;53:370-418.
15 +
Phillips LD: Bayesian Statistics for Social Scientists . New York, Crowell, 1974;.
16 +
Donner A:  A Bayesian approach to the interpretation of subgroup results in clinical trials . J Chronic Dis 1982;;35:429-435.
17 +
Pater JL, Willan AR:  Clinical trials as diagnostic tests . Controlled Clin Trials 1984;;5:107-113.
18 +
Thomas DC, Siemiatycki J, Dewar R, et al:  The problem of multiple inferences in studies designed to generate hypotheses . Am J Epidemiol 1985;; 122:1080-1095.
19 +
Mushlin AI:  Diagnostic tests in breast cancer: Clinical strategies based on diagnostic probabilities . Ann Intern Med 1985;;103:79-85.
20 +
 Lipid Research Clinics Program: The Lipid Research Clinics Coronary Primary Prevention Trial results: I. Reduction in incidence of coronary heart disease . JAMA 1984;;251:351-364.
21 +
MacMahon B, Yen S, Trichopolous D, et al:  Coffee and cancer of the pancreas . N Engl J Med 1981;;304:630-633.
22 +
Jick H, Dinan BJ:  Coffee and pancreatic cancer . Lancet 1981;;2:92.
23 +
Goldstein HR:  No association found between coffee and cancer of the pancreas . N Engl J Med 1982;;306:997.
24 +
Wynder EL, Hall NEL, Polansky M:  Epidemiology of coffee and pancreatic cancer . Cancer Res 1983;;43:3900-3906.
25 +
Kinlen LJ, McPherson K:  Pancreas cancer and coffee and tea consumption: A case-control study . Br J Cancer 1984;;49:93-96.
26 +
Gold EB, Gordis L, Diener MD, et al:  Diet and other risk factors for cancer of the pancreas . Cancer 1985;;55:460-467.
27 +
Hsieh C, MacMahon B, Yen S, et al:  Coffee and pancreatic cancer (chapter 2) . N Engl J Med 1986;;315:587-589.
28 +
Frieman JA, Chalmers TC, Smith H Jr, et al:  The importance of beta, type II errors and sample size in the randomized control trial: Survey of 71 'negative' trials . N Engl J Med 1978;;299:690-694.
29 +
Young MJ, Bresnitz EA, Strom BL:  Sample size nomograms for interpreting negative clinical studies . Ann Intern Med 1983;;99:248-251.
30 +
Sackett DL:  Bias in analytic research . J Chronic Dis 1979;;32:51-63.
31 +
Winkelstein W Jr:  Smoking and cancer of the uterine cervix: Hypothesis . Am J Epidemiol 1977;;106:257-259.
32 +
Wright NH, Vessey MP, Kenward B, et al:  Neoplasia and dysplasia of the cervix uteri and contraception: A possible protective effect of the diaphragm . Br J Cancer 1978;;38:273-279.
33 +
Harris RWC, Brinton LA, Cowdell RH, et al:  Characteristics of women with dysplasia or carcinoma in situ of the cervix uteri . Br J Cancer 1980;;42:359-369.
34 +
Lyon JL, Gardner JW, West DW, et al:  Smoking and carcinoma in situ of the uterine cervix . Am J Public Health 1983;;73:558-562.
35 +
Godfrey K:  Comparing the means of several groups . N Engl J Med 1985;;313:1450-1456.
36 +
Cupples LA, Heeren T, Schatzkin A, et al:  Multiple testing of hypotheses in comparing two groups . Ann Intern Med 1984;;100:122-129.
37 +
Cole P:  The evolving case-control study . J Chronic Dis 1979;;32:15-27.
38 +
Fleiss JL: Statistical Methods for Rates and Proportions , ed 2. New York, John Wiley & Sons Inc, 1981;, p 14.
39 +
Rothman K:  A show of confidence . N Engl J Med 1978;;299:1362-1363.
40 +
Browner WS, Newman TB:  Confidence intervals . Ann Intern Med 1986;;105:973-974.

Letters

Submit a Letter
CME Course for:


You need to register in order to view this quiz.


To understand the clinical management of acute heart failure syndromes.
Accreditation Information The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
To view and print your certificate and access a summary of your CME courses go to My CME.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s “Cited By” API will populate this tab (http://www.crossref.org/citedby.html).
Compression-Only CPR: Pushing the Science Forward
Cone
AAM 2010;304:1493-1495.
All you need to read in the other general journals
BMJ 2010;341:c5893-c1494.
Submit a Response

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

© 2012 American Medical Association. All Rights Reserved.
Powered bySilverchair Information Systems