Quality Improvement Organizations and Hospital CareQuality Improvement Organizations and Hospital Care

To the Editor: In their study of the effectiveness of QIOs in improving hospital care, Drs Snyder and Anderson¹ reported that hospitals in the state of Washington that participated with our QIO improved more than nonparticipating hospitals for 11 of 13 quality indicators. Although the study did not have adequate power to evaluate the statistical significance of the individual measures at a state level, the consistent direction of the differences is highly unlikely to have occurred by chance alone (P = .011, based on a binomial distribution). As much as we would welcome the opportunity to cite this finding as evidence of the effectiveness of our work, serious methodological flaws in the study render the finding nearly meaningless.

The authors wrote that the follow-up sample of records was collected “toward the end of the QIOs’ contract cycle that ended in 2002.” While accurate, the relevant period for evaluating the effectiveness of efforts to improve care would have been the time period during which the care was delivered, not the timing of data collection.

The quality indicator data we provided to Snyder and Anderson included care delivered only during the first 17 months of the 36-month QIO intervention period. The data regarding hospital participation, however, covered the entire 36-month period. Thus, the data they analyzed reflected less than one half of the time period corresponding to the QIO interventions they were attempting to evaluate. Moreover, the “participating hospitals” group could include hospitals whose participation began after the end of the “follow-up” data time frame. We believe that the data set the authors analyzed could not be used to reach their principal conclusion, as the effectiveness of a 3-year effort cannot be judged using information from only the first 17 months of that period.