Administrative data offer impressive amounts of information that can be readily analyzed. However, how credible are the results derived from these behemoth data sets? Moreover, how prudent is it to translate those results into policy actions, and, if this is done, what should that translation path involve?
In this issue of JAMA, Joynt and colleagues1 report the results of an interesting analysis that exemplifies these challenges. The authors used administrative data from almost 10 million admissions of Medicare patients in 2002-2010 to evaluate the change in 30-day mortality rates for acute myocardial infarction, congestive heart failure, and pneumonia in critical access hospitals (CAHs) vs non-CAHs. Critical access hospitals fared worse by 0.3% per year, such that by 2010, CAHs had higher mortality rates compared with non-CAHs (13.3% vs 11.4%). The nominal statistical significance also was maintained in analyses matching CAHs against other rural non-CAHs, although the absolute difference in mortality declined to 0.1% (95% CI, 0.0%-0.2%) per year. This means that only a small portion of the variation in mortality risk was explained by CAH status. Numerous sensitivity analyses also demonstrated a relatively consistent pattern.