Relationship of Primary Care Physicians' Patient Caseload With Measurement of Quality and Cost Performance FREE

Ample evidence reveals that despite high and rising costs of health care in the United States, quality is lagging.^1- 2 Moreover, research has repeatedly documented considerable variation in Medicare spending and quality across the country independent of patient illness levels or demographic characteristics.^3- 4

To stimulate improved quality and lower costs of ambulatory care for its beneficiaries, the Centers for Medicare & Medicaid Services has been overseeing a series of value-based purchasing initiatives.⁵ Among them are pay-for-performance demonstration projects and the Physician Quality Reporting Initiative,^6- 7 which could act as blueprints to inform broader initiatives that are purposed to strengthen accountability among physicians participating in the Medicare program. However, physicians in many specialties, particularly primary care physicians, provide a wide variety of services to different patients. It is unlikely that individual primary care physicians annually see a sufficient number of eligible patients to produce statistically reliable performance measurements on common quality and cost measures, calling into question whether their performance can be differentiated with respect to national benchmarks.⁸

This limitation might be overcome by measuring the collective performance of primary care physicians at the practice in which they work.^9- 10 To investigate this possibility, we measured the performance of primary care physicians by accessing 2 resources: (1) a national random sample of Medicare beneficiaries and (2) a database identifying the practice affiliations of the physicians who delivered ambulatory care to these Medicare patients.

Data preparation and analyses required 4 steps. First, we calculated national estimates of costs and quality, which were used to calculate the caseloads necessary to detect relative differences in performance for each measure. Next, we linked primary care physicians who billed fee-for-service Medicare to their physician practices. We then identified Medicare beneficiaries with office visits to each primary care physician to generate caseloads. Finally, we estimated the proportion of primary care physician practices with sufficient caseloads to detect relative differences in performance.

To calculate the national mean Medicare costs for ambulatory care, we used reimbursement from the Part B 5% sample as part of the 2005 continuous Medicare history sample. National rates of process and outcome measures of quality were calculated among beneficiaries from the 2005 Part B 20% sample who were at least 65 years old and continuously enrolled in fee-for-service Medicare. Process measures of quality were quality indicators identified in Part B claims data—mammography for women 65 to 69 years old and hemoglobin A_1c testing for individuals with diabetes 65 to 75 years old.¹¹ Outcome measures of quality were coded from 2005 Medicare provider analysis and review (MedPAR) claims data among beneficiaries from the Part B 20% sample. These measures included preventable hospitalization associated with any of 13 adult ambulatory care sensitive conditions as well as hospital readmission for any reason within 30 days of discharge for congestive heart failure (CHF) because of the relatively high frequency of this event among individuals who are older than 65 years.^12- 13

Rates of quality indicators were based on different eligible patient subpopulations. Because beneficiaries enter the Medicare program at different points during the year at age 65, the denominator for rate of mammography was restricted to women aged 66 to 69 years and receipt during 2005 was included in the numerator. Likewise, the denominator for rate of hemoglobin A_1c testing was restricted to patients with diabetes aged 66 to 75 years who were denoted with a diabetes International Classification of Diseases, Ninth Revision (ICD-9) code affixed to the earlier of either: (1) a claim from an inpatient stay or emergency department visit or (2) two ambulatory evaluation and management visits in 2005. Receipt of a hemoglobin A_1c test was counted in the numerator. To give patients with diabetes a full year to obtain a hemoglobin A_1c test, its receipt was checked 1 year from the date of identification of diabetes into 2006. For preventable hospitalization rate, the denominator represented the total number of beneficiaries and any such admission that was not a transfer was counted in the numerator. All hospital admissions for CHF in the first 11 months of 2005 represented the denominator for 30-day readmission rate; the numerator included each readmission within 30 days for any reason, excluding transfers.

Physicians included in this study were limited to primary care physicians—defined as internists, family practitioners, general practitioners, or geriatricians—according to an algorithm that assigned a specialty based on the specialty code appearing on a plurality of the physicians' claims from the 2005 Part B 20% sample. To avoid including primary care physicians who saw few Medicare patients, those who submitted ambulatory evaluation and management claims for fewer than 10 unique patients from the 20% sample were excluded.

Eligible primary care physicians were linked via unique physician identification number to their organizational affiliations in the Healthcare Organization Services database, which is maintained and continually updated by IMS Health using standardized source/verification procedures.¹⁴ The Healthcare Organization Services database contains comprehensive data on physicians, their affiliated organizations, and the structural relationships between those organizations.

Primary care physicians with 1 affiliation to a physician practice were included in the analyses. A physician practice is defined as a site of 1 or more physicians sharing office space and staff or if 2 or more sites are corporately owned, a physician practice encompasses all such sites. Primary care physician practices were categorized by size according to the number of Medicare-matched primary care physicians working in the same physician practice.

Caseloads were estimated for individual primary care physicians and their respective practices. They were based on all eligible Medicare beneficiaries in the 2005 Part B 20% random sample with an ambulatory evaluation and management visit in Part B claims, or at a rural health clinic or federally qualified health center in outpatient claims. Although only the 20% sample is available to researchers, it contains hundreds of thousands of beneficiaries. We multiplied caseloads by 5 to better approximate the true number of Medicare beneficiaries receiving care from each primary care physician. Each beneficiary was counted as part of a caseload if he or she had at least 1 office visit with a primary care physician. Beneficiaries were counted once per primary care physician practice but could be counted in the caseloads of more than 1 primary care physician practice. We did not assign each beneficiary to a unique primary care physician or practice because our purpose was to measure the potential for measuring performance, not performance per se. The results, therefore, reflect a conservative bias by overestimating the number of beneficiaries able to be measured per primary care physician practice.

To determine the proportion of primary care physician practices with enough patients to detect a 10% relative difference in each performance measure, we used national mean costs and quality measure rates as the basis for performing sample size calculations. We assumed that a difference in performance of 10% from the national mean or rate represented a meaningful difference; therefore, a 1-sample, 2-tailed test was used to determine mean costs and the equivalent test was used for proportions in quality measures. The type I error rate was set at 5% (α = .05) with a power of 80% (1 − β = 0.8); costs, n = [(1.96 + 0.84)σ / (μ₁ − μ₀)]²; quality, n = {1.96[π₀(1 − π₀)]^0.5 + 0.84[π₁(1 − π₁)]^0.5} / (π₀ − π₁).² Finally, the proportion of primary care physician practices that met the minimum caseload for each performance measure was calculated. We also determined the national proportion of patients who visited practices with sufficient caseloads for each performance measure. As a sensitivity analysis, we doubled and tripled caseloads to simulate 2- and 3-year caseloads.

All analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina).

Among 151 190 primary care physicians billing at least 1 Medicare claim in 2005, 115 984 (77%) saw at least 50 patients (Figure 1). A total of 89 874 primary care physicians (77%) matched to at least 1 physician practice in the Healthcare Organization Services database. Primary care physicians with no match may have spent a small amount of their time in clinical practice or may have billed Medicare from a hospital or a hospital-based clinic outside the scope of the definition of a physician practice. They may also not be accounted for in the Healthcare Organization Services database because they were billing Medicare for the first time. Approximately 80% of matched primary care physicians were affiliated with a single physician practice. We excluded primary care physicians affiliated with more than 1 practice, giving a final sample of 71 980 primary care physicians. Compared with their counterparts with multiple practice affiliations, primary care physicians with 1 affiliation saw the same number of patients annually.

As shown in Table 1, of the 71 980 primary care physicians affiliated with 30 794 physician practices, most (60.6%) were solo practitioners. The largest physician practices were the least common (0.2%) but included almost 8% of primary care physicians.

Table 2 shows the median annual caseloads of all patients and of patients eligible for different performance measures. Caseloads ranged from a median of 170 patients (IQR, 100-275) for solo practitioners to 13 400 patients (IQR, 10 620-16 885) for practices with more than 50 primary care physicians. Median caseloads incrementally increased for patients eligible for different quality measures as practices increased in size. The national median caseload of all Medicare patients in primary care physician practices was 260 patients (IQR, 135-500) and was no larger than a median of 30 patients for any specific quality measure.

Table 3 shows mean national ambulatory costs and rates of quality measures. A caseload of 2526 patients would be required to reliably detect a 10% relative difference in ambulatory costs. The caseload required to detect a 10% relative difference in quality measures was lowest for mammography at 328 eligible women and highest for preventable hospitalization at 19 069 patients.

Table 4 shows the percentage of primary care physician practices with sufficient caseloads to detect a 10% relative difference in each performance measure. With a 1-year caseload, virtually no practices with fewer than 6 primary care physicians had enough patients to reliably detect a 10% relative difference in costs or in any quality measure. Approximately 9% of practices with 6 to 10 primary care physicians had enough patients to reliably detect a 10% relative difference in costs, but less than 3% of primary care physician practices could do so with mammography or hemoglobin A_1c testing. Approximately 50% of practices with 11 to 20 primary care physicians had enough patients to reliably detect a 10% relative difference in costs, but fewer than 30% of primary care physician practices could do so for any quality measure. More than half of practices with 21 to 50 primary care physicians and all practices with more than 50 primary care physicians had enough patients to detect 10% relative differences in costs, mammography, and hemoglobin A_1c testing. No primary care physician practices had sufficient caseloads to detect relative differences for preventable hospitalization or 30-day readmission after discharge for CHF.

Nationally, however, any performance measure could be reliably measured for less than 2% of all primary care physician practices since most consisted of a small number of primary care physicians. When we simulated caseloads over 2- and 3-year periods, the proportion of primary care physician practices nationally with sufficient caseloads to reliably measure performance did not exceed 12% for costs or any quality measure.

Finally, a minority of Medicare patients visited primary care physician practices with sufficient caseloads to reliably measure costs and quality. As shown in Figure 2, the proportion of such patients was no more than 20% using a 1-year caseload; this proportion increased with a 3-year caseload to almost 50% for costs, approximately 40% for women eligible for mammography, and about 35% for patients with diabetes eligible for hemoglobin A_1c testing.

We found that roughly 65% of all primary care physicians active in the Medicare program work in practices with insufficient numbers of beneficiaries to reliably differentiate their practices' performance from national quality and cost benchmarks. Only the largest primary care physician practices, which are also the most uncommon, can be expected to have sufficient caseloads to measure significant differences in performance.

To the best of our knowledge, ours is the first study to estimate the percentage of primary care physicians who care for sufficient numbers of Medicare patients to produce statistically reliable performance measurements at the practice level. Other studies have assessed statistical reliability based on the probability that a physician's performance is the same after repeated measurements.^15- 17 We used power calculations instead because they are less prone to misclassification bias and better aligned with the goal of value-based purchasing, which seeks to differentiate performance between physicians. If a sound majority of primary care physicians at the practice level do not have caseloads large enough to detect 10% relative differences in performance of common cost and quality measures from national benchmarks, then our findings have several implications for value-based purchasing.

Our study suggests that rethinking the approach to performance measurement in ambulatory care may be necessary for the Medicare program. Many existing quality measures are already limited by their focus on particular subpopulations of patients, such as patients with diabetes aged 65 to 75 years and women aged 65 to 69 years. Preventable hospitalization and readmission are relatively rare events, and a multiplicity of factors outside of ambulatory care can influence their occurrence.¹⁸ Cost measures always require large sample sizes because individual patients' annual expenditures vary widely, even after risk adjustment.¹⁹

Overcoming these limitations is possible by increasing the number of patients eligible for statistical analysis. One approach would be to pool patients from all payer sources. Another way to increase sample size would be to pool patients across a variety of measures rather than for each single measure, although a physician's performance on different ambulatory care measures is poorly correlated.^20- 22 Alternatively, performance could be measured over a 2- or 3-year period, although this approach would not provide timely information on performance. New performance measures not available from claims data, such as patient experience or work processes measures, might be devised but would require the expense of collecting data.

In addition to the current limitations of performance measurement, an interrelated challenge for value-based purchasing is identifying a model of accountability without excluding large portions of physicians and the patients receiving their care. The medical home has recently emerged as an attractive means to explicitly identify an accountable entity.²³ Its most basic form would involve a prespecified clinician or set of clinicians responsible for a patient's care. Yet, our findings suggest that it would be extremely difficult to surmount the limitations of performance measurement for a medical home the size of the typical primary care physician practice.

Therefore, the right model for measuring performance and fostering accountability may vary in accordance with the different ways in which physician practices are organized. We found that it is possible to reliably measure 10% relative differences in performance for the care of Medicare patients in practices with 50 or more primary care physicians. Since few primary care physicians practice in groups of this size, it might be possible if, for value-based purchasing purposes, smaller primary care physician practices were aggregated into virtual groups or networks of more than 50 primary care physicians. It has been suggested that organizations such as independent physician associations, physician-hospital associations, or accountable entities centered around the acute care hospital in which physicians admit the majority of their patients could serve as virtual groups.^24- 26 However, at present, there appear to be few independent physician associations and physician-hospital associations in which physicians are successfully working together to improve quality,²⁷ and other than physician-hospital associations, accountable entities centered around acute care hospitals currently do not exist. A virtual group, therefore, would only be effective insofar as its physicians believed that they shared responsibility for working together to improve the care they provide, even though they remain in independent practices that are smaller than the virtual group.

Our study has several limitations. First, accurately matching physicians to their practices in a given year is challenging because physician turnover makes matching physicians to practices a perennial moving target. To be more confident about practice assignment, we excluded primary care physicians affiliated with multiple practices; these primary care physicians had the same caseloads as primary care physicians with a single-practice affiliation. It is unlikely that we inaccurately assigned a disproportionate number of primary care physicians to the smallest practices since the percentage of primary care physicians who were solo practitioners was lower than national survey estimates.²⁸ If we inaccurately assigned a higher proportion of physicians to the largest practices, then any bias in our estimate of the physician distribution would overestimate the number of physicians affiliated with practices that could reliably detect relative differences in performance.

Second, counting all patients treated by primary care physicians in a physician practice, as opposed to assigning patients to unique practices, allowed us to calculate a conservative estimate of the proportion of practices of various sizes that could reliably detect relative differences in costs and quality. Had we assigned each patient to a unique practice, an even smaller number of primary care physician practices would have met the necessary caseload thresholds. Likewise, we did not account for clustering of patients within practices.

Third, larger groups of primary care physicians may be desirable for performance measurement, but there may be advantages to patients receiving care in smaller physician practices.

Finally, we chose a 10% relative difference in performance from national rates as representing a meaningful difference. We thought this level was an appropriate starting point for measuring costs and quality for 2 reasons: first, detecting smaller differences would require more patients and make it more challenging to accurately differentiate practices. Second, increases or decreases in costs or quality greater than 10% of a national average may be unrealistic. Power calculations could use larger relative differences for rare events such as preventable hospitalization and a log-transformation to lessen the variability of ambulatory costs. Such approaches would reliably differentiate more primary care physician practices, yet the proportion of primary care physician practices with insufficient caseloads would remain considerable.

In the absence of performance measurement approaches that amass larger numbers of eligible patients at the physician or practice level, the results from this study call into question the wisdom of pay-for-performance programs and quality reporting initiatives that focus on differentiating the value of care delivered to the Medicare population by primary care physicians. Novel measurement approaches appear to be needed for the twin purposes of performance assessment and accountability.