Statistical Association and Causation: Title and subTitle BreakContributions of Different Types of Evidence

Advances in medical knowledge proceed on several fronts, optimally simultaneously. Each discipline provides unique, relevant, and complementary information to a totality of evidence. When the totality of evidence is sufficient, health care professionals can make the most rational decisions for individual patients and policy makers can make the most rational decisions for the health of the general public.¹ When the totality of evidence is incomplete, it is appropriate to remain uncertain.² Nonetheless, health care professionals and policy makers are always faced with decision making. Although medical researchers are likely to be familiar with these concepts, this commentary is primarily for clinicians and policy makers to increase their knowledge and understanding of the unique contributions of different types of evidence to the conclusion of a valid statistical association as well as the need to evaluate the totality of evidence to judge causality.

Basic Research. Basic research provides cogent biological mechanisms to answer the crucial question of why an agent or intervention affects morbidity or mortality. Basic research has the unique strength of precision and can control virtually all genetic and environmental exposures to provide important directions to guide as well as explain human research. Basic research also has the unique disadvantage of questionable relevance to humans.

Clinicians and Clinical Research. Clinicians provide enormous benefits to patients by applications of advances in diagnosis and treatment and formulate hypotheses based on their clinical experiences, specifically case reports and case series. Clinical investigators test the relevance of basic research to healthy and sick individuals.

Epidemiology and Biostatistics. Epidemiologists and biostatisticians, optimally collaborating with clinicians, formulate hypotheses from descriptive studies and test those hypotheses in analytic studies. Analytic epidemiological studies designed a priori to test hypotheses address the complementary question of whether there is a valid statistical association between an exposure, such as a drug or intervention, and an outcome, such as morbidity or mortality. Epidemiology is relevant to free-living humans but has the unique disadvantage of being crude and inexact as observations on free-living humans can never take place under the controlled conditions of basic research.

Descriptive Studies. Descriptive studies report patterns of disease in relation to person, place, and time. Data from descriptive studies are essential for public health administrators to allocate resources efficiently and to plan effective prevention or education programs, and for researchers to formulate hypotheses that can be tested subsequently using an analytic study design. The 3 main types of descriptive studies are correlational or ecologic, which consider patterns of disease among populations, case reports or case series, and cross-sectional surveys, both of which involve individuals.

Observational Analytic Studies. Observational analytic studies can be either case-control studies, especially useful for rare diseases, or cohort studies, especially useful for rare exposures. These are hypothesis testing for moderate to large effect sizes only if designed a priori to do so. If not, the results should instead be considered hypothesis formulating. In particular, case-control or cohort study data collected for administrative purposes, no matter how large in sample size, should be considered hypothesis formulating. For small to moderate effect sizes, however, all observational analytic studies, no matter how well designed, conducted, analyzed, and interpreted, are hypothesis formulating as their inherent uncontrolled and uncontrollable confounding can be as big as the effect sizes.

Meta-analyses of Randomized Trials. Meta-analyses of randomized trials that were not designed a priori to test hypotheses about clinical events and, as a result, were not large enough to do so, should be considered hypothesis formulating. Meta-analysis of individual randomized trials reduce the role of chance but may introduce bias and confounding. For example, individual trials may have different design features, different exposures being studied, including the nature and dose of drugs, and different rates of adherence and follow-up. The quality and usefulness of any meta-analyses depend on the quality and comparability of the data from the component trials. In particular, to be considered hypothesis testing the component trials should have been designed and conducted to evaluate the end point of interest, should have high adherence and follow-up rates, and should have reasonably comparable exposures and outcomes.

Large-Scale Randomized Trials. Large-scale randomized trials designed a priori to test hypotheses provide the most reliable evidence concerning the most plausible small to moderate effect sizes. In large samples, randomization provides a degree of control of confounding that is not possible to achieve in any observational analytic study.³

The results of any analytic epidemiological study designed a priori to test a hypothesis may have 3 plausible alternative explanations. The first is the play of chance or the “luck of the draw,” which can occur any time a sample of a population is examined. The second is bias or systematic error in the way individuals were selected or observed in the study. The third is confounding or variables associated with the exposure and independent risk factors for the outcome. All 3 explanations must be carefully considered before concluding that there is a valid statistical association from any analytic study designed a priori to test a hypothesis.

Judgments about causation can be safely made only on a sufficient totality of evidence. Several positive criteria support a judgment of causality, including strength of association, biological credibility, consistency, temporal sequence, and dose-response relationship.³

A greater strength of association implies that plausible alternative explanations are less likely. A known or postulated biological mechanism also enhances the belief in causality. In addition, the belief in causality is enhanced by the consistency of findings from independent studies designed a priori to test a hypothesis. However, temporal sequence can sometimes be difficult to establish. Many lifestyle variables such as physical activity, smoking, or consumption of particular foods or beverages may be altered after the first symptoms appear, either deliberately or as a biological consequence of the disease. In addition, a dose-response relationship may merely reflect the effect of an uncontrolled confounding factor, as is likely to be the case with cigarette smoking and cirrhosis of the liver, a statistical association most likely due to confounding by heavy alcohol consumption.⁴ Finally, a dose-response relationship can be missed by an inability to precisely measure exposure.

The conclusion of a valid statistical association can be safely made from an analytic study designed a priori to test a hypothesis after exclusion of chance, bias, and confounding as plausible alternative explanations. For small to moderate effect sizes, large-scale randomized trials are a necessary component of a sufficient totality of evidence. A valid statistical association may or may not be causal. Causation is a judgment based on the totality of evidence that includes several positive criteria. As Hill aptly stated in 1965: “All scientific work is incomplete—whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time. Who knows, asked Robert Browning, but the world may end tonight? True, but on available evidence most of us make ready to commute on the 8:30 next day.”⁵