Association Between Licensing Examination Scores and Resource Use and Quality of Care in Primary Care Practice FREE

PRIMARY CARE physicians are the foundation of westernized health care. They manage episodic and chronic illness, play a central role in instituting interventions for primary and secondary disease prevention, prescribe and monitor the majority of drug treatment, and function as the entry point to costly health care resources.^1- 7 The quality of care delivered by primary care physicians and their use of resources can have a considerable impact on the cost-effectiveness of health care delivery.³ Preventive care,^8- 11 prescribing,^12- 20 and chronic disease management^21- 28 have been identified as areas in which quality of care could be improved.

Continuing medical education has been identified as a solution to this problem because low self-reported rates of preventive care are associated with inadequate knowledge of preventive guidelines,^29- 30 and improvements in prescribing and chronic disease management have been reported with disease-specific education in diagnosis and management.^31- 32 Yet mandatory continuing medical education requirements have not been associated with improvements in practice,^33- 34 and this observation has likely contributed to a call for mandatory assessment of the competence of practicing physicians through relicensure and recertification examinations.^35- 36

The question that has not been adequately addressed is whether tests that purport to measure clinical competence (licensing and certifying examinations) actually predict performance in subsequent practice. Ramsey et al,³⁷ who were the first investigators to address this question, found that scores on the American Board of Internal Medicine certification examination predicted colleagues' ratings of the quality of care delivered by internists 5 to 8 years after certification. Although these results are promising, we do not know whether they are generalizable to family physicians and general practitioners, physicians who provide the majority of primary care, or if colleagues' ratings actually predict what is done in practice. The possibility that licensing examination scores may predict performance in practice needs to be examined because interventions to improve practice through licensure and certification provide the ultimate opportunity for an ounce of prevention to be worth a pound of cure.

Clinical competence has not only been considered to be relevant to the quality of care delivered, but it has also been associated with variation in the use of resources by primary care physicians. Two studies, one British³⁸ and one American,³⁹ have found that primary care physicians with higher self-reported competence were more likely to refer their patients for specialty consultation. This is relevant because referral is the main mechanism by which patients gain access to hospital resources and expensive surgical and investigative procedures,³ and unexplained differences in referral rates exist among primary care physicians.^40- 43 Considerable variation also exists among primary care physicians in prescription rates,^12,44- 49 and it has been suspected that these differences are related to clinical competence.^50- 51 In particular, higher rates of prescription of medications often used for the relief of undifferentiated symptoms^49,52- 60 in the elderly (eg, benzodiazepines, nonsteroidal anti-inflammatory drugs [NSAIDs], analgesics) are seen among physicians who have low rates of prescribing for disease-specific treatments (eg, hypoglycemics, cardiac medications).^46- 47 It has been postulated that this pattern is related to an underdiagnosis and undertreatment of disease because of poorer skills in diagnosis and management. There has been no direct test of this association.

Several methodological problems have limited research on the relationship between licensing and certification examination scores and practice. First, physicians with higher scores are more likely to participate in follow-up studies.⁶¹ Second, mandatory examinations have traditionally been completed at the end of medical school, with only selected groups of physicians going on to take voluntary certification examinations. Thus, to include all physicians, evaluation would have to take place at a point in training when competence in diagnosis and management would not be fully developed. Finally, follow-up studies of physicians into practice are expensive because assessment of 50 to 200 patients may be needed to obtain stable estimates of performance, and the alternate of self-reported activity is not a valid assessment of what physicians actually do in practice.^62- 63

An opportunity to overcome 4 major methodological difficulties existed in Québec, a Canadian province with a population of 7.2 million people, 4 medical schools, and approximately 17000 physicians. First, in this province, mandatory assessment of a physician's competence is carried out shortly before an individual enters practice. To be eligible for licensure, physicians must complete a family medicine or other specialty residency and then pass a combination licensing and specialty certification examination. Second, physicians are tested in specific areas of competence and must meet minimum standards in each area to be eligible for licensure. As a result, the subset of competencies that would be expected to have the strongest link with specific practice activities could be assessed. In addition, Québec was the first jurisdiction to include an objective structured clinical examination (OSCE) in its licensure examination process.⁶⁴ This provided the opportunity to evaluate the utility of this new form of testing of clinical skills. Third, all medical services provided to provincial residents are covered by a universal health insurance plan with 1 government agency responsible for payment of all physician services. This provided the opportunity to obtain a complete census of the practice activity of fee-for-service physicians. Fourth, the availability of high security encryption technology and third-party linkage provided methods of protecting the confidentiality of individual physicians so that all physicians could be included in a study. Although differences in licensing examination requirements and medical service data acquisition exist in Québec, undergraduate education and family medicine training are similar to other accredited North American programs.

In the current study, we used this opportunity to evaluate several aspects of primary care in which variation in care had been identified and in which practice activity could be measured using information retained in provincial health care databases. In relationship to quality of care, we tested the hypotheses that better clinical skills and prevention knowledge would be related to higher rates of mammography screening and that higher scores in diagnosis and management would be related to lower rates of potentially inappropriate prescribing. We included the assessment of clinical skills as a potential predictor of mammography screening because of the reported association between physical examination and preventive care.⁶⁵ In relationship to resource use, we tested the hypotheses that higher overall competence would be related to higher consultation rates and that higher scores in diagnosis and management would be related to lower rates of symptom-relief prescribing relative to disease-specific prescribing.

A prospective follow-up study of a cohort of newly licensed family physicians was conducted. Physicians were eligible for inclusion in the study if they had passed the family medicine licensing examination in 1991, 1992, or 1993; applied for a license to practice in Québec; and entered fee-for-service practice. Physicians paid by salary were excluded because there was no accurate way to identify all patients who had been seen by these physicians even though all prescriptions and referrals for consultation and mammography could be ascertained. Potentially eligible physicians were identified by the Québec College of Physicians, Montreal, Québec, and the license numbers of these physicians were provided to the provincial health insurance agency (Régie de l'Assurance-Maladie du Québec [RAMQ], Québec City, Québec). Practice activity was assessed in the 18-month period after the earliest possible entry into practice. For each cohort, the follow-up period was between July 1 of the examination year and December 31 of the following year.

Candidates applying for licensure in Québec must pass the College of Family Physicians of Canada Certification Examination (CFPC_Ex) and the Québec OSCE.^64,66 The CFPC_Ex assesses 4 areas of competence: diagnosis, management, prevention, and communication.⁶⁶ Diagnosis, management, and prevention are measured by multiple-choice questions and short-answer management problems and communication is measured by a simulated patient office oral examination. To pass the CFPC_Ex, a candidate must obtain a score of at least 60% each in diagnosis, management, and communication and a score of at least 50% in prevention. The OSCE primarily measures clinical skills by direct, structured observation of a candidate's performance in 24 to 26 clinical problems.⁶⁴ Clinical problems are presented by standardized patients and performance is assessed by physician examiners. Scores for each case are averaged to produce an overall OSCE score. Candidates fail the OSCE if their score is more than 2 SDs below the mean. Candidates who pass some but not all components of the 2 examinations are reviewed by the licensing examination board and decisions are made on a case-by-case basis.

The subscores from the CFPC_Ex and the OSCE and an overall average score (the mean of all scores) were retrieved for eligible physicians. There were 6 administrations of the licensing examination between 1991 and 1993. To adjust for potential differences in the difficulty of the examination, scores were standardized using the reference-group method.⁶⁷ A large homogeneous group of examinees (first-time takers from North American medical schools) is assumed to have similar ability over time and this group was used as the reference standard. Reliability of scores was estimated in each year and summarized across years using a weighted intraclass correlation coefficient.⁶⁸ Standardized examination scores for each physician and their license number were sent to the RAMQ. Personal identifiers on the score file and health care databases were encrypted to protect confidentiality but permit database linkage.

Data Source. Practice activity and practice population characteristics were assessed using information from 4 provincial health care databases and the national 1992 census data. The physician claims file provides, for all medical services delivered on a fee-for-service basis (95% of services⁶⁹ ), information on the treating physician; referring physician; recipient; type, location, and date of service; and the diagnosis for the visit. The prescription claims file provides information on all prescription drugs dispensed to the elderly, including the recipient, prescribing physician, drug, quantity dispensed, and dispensing date. These reimbursement files are systematically audited and have been independently validated.^70- 71 The registrant database contains the registration number, age, sex, residential address, and health district of each provincial resident. The master physician file contains information on the license number, specialty, year of graduation, medical school, age, and sex of Québec physicians. The national 1991 census data⁷² were linked to the registrant database to provide information on the average family income and mean educational achievement in the health district of each registrant.

Data Retrieval. A 3-step process was used to obtain practice data for each physician. First, the RAMQ identified all registrants who had been seen on a fee-for-service basis by each study physician in the 18 months following their earliest possible entry date into practice. Second, all services and prescriptions provided to these patients by the study physician as well as other physicians in the province were retrieved from the 18-month follow-up window established for each physician. Third, all services and prescriptions provided to these patients were retrieved from the 18-month period before the follow-up window. These baseline data were used to account for differences in the practice population characteristics that existed prior to the first contact with the study physicians and, thus, could not be attributed to their practice patterns.

Practice activity for each physician was measured in 3 practice populations: all outpatients, elderly patients, and women aged 50 to 69 years. Patients were considered to be in a physician's practice population if they received 1 or more services from the study physician in an outpatient setting (emergency department, outpatient clinic, or office practice). For women to be considered eligible for mammography screening, we further restricted the practice population to women seen in an outpatient setting where the institution of preventive care would normally be expected. The setting in which care was delivered was determined by the location recorded on the billing claim, and age restrictions, when relevant, were determined by the age recorded for an individual in the registrant database. Each practice population denominator was composed of the number of different patients seen by a physician during the follow-up period. Patients who were seen by multiple study physicians were represented in more than 1 physician's practice population. Four practice indicators were generated for each physician: consultation rate, symptom-relief prescribing rate compared with disease-specific prescribing rate, inappropriate prescribing rate, and mammography screening rate in women aged 50 to 69 years.

Consultation rate was defined as the proportion of outpatients in a physician's practice who were referred at least once for a specialty consultation by the study physician. A patient was designated as having been referred to the specialist when the study physician was identified by the specialist as the referring physician on the claim for a consultation.

Symptom-relief prescribing rate relative to disease-specific prescribing rate was defined as the difference between the proportion of elderly patients for whom the study physician prescribed disease-specific medication and the proportion of elderly patients who were prescribed symptomatic medication by the study physician. Symptomatic medication, defined as drugs that relieve symptoms but have little impact on the disease process,⁴⁷ included NSAIDs, benzodiazepines, and low-dose narcotic analgesics. Disease-specific medication, defined as drugs that would likely be used exclusively to treat an investigation-confirmed disease state,⁴⁷ included anticoagulants, anticonvulsants, antidepressants, antihypertensives, anti–Parkinson disease medication, asthma preparations, corticosteroids, diuretics, antiglaucoma medication, cardiac medication, hypoglycemic medication, and thyroid and antithyroid medication. (The list of medications included in symptom-relief and disease-specific prescribing rates is available on request from the authors.) The list of disease-specific and symptomatic medications was created using the classification established by McGavock et al,⁴⁷ excluding drugs that could be obtained as over-the-counter products. The prescription of drugs started by other physicians and refilled by the study physician was distinguished from the prescription of drugs started by the study physician (no evidence of a previous prescription for the drug in the 6 months preceding the study physician's prescription) for both disease-specific and symptom-relief medication.

Inappropriate prescribing rate was defined as the proportion of patients prescribed a relatively contraindicated psychotropic drug, cardiovascular drug, or NSAID. Inappropriate drugs were identified from a published international expert consensus⁷³ and were defined as medications that should be avoided in the elderly because of the risk of toxic effects (long-acting benzodiazepines, meprobamate, barbiturates, amitriptyline, indomethacin, phenylbutazone, propoxyphene, pentazocine, reserpine, methyldopa, and propranolol). Prescriptions of inappropriate drugs that were started by other physicians and refilled by the study physician were distinguished from drug prescriptions that were started by the study physician without evidence of a previous prescription in the past 6 months.

Mammography screening rate in women aged 50 to 69 years was defined as the proportion of eligible women aged 50 to 69 years who received bilateral mammography at the request of the study physician. Eligible women were those who had no diagnosis of breast cancer or breast disease or a diagnostic mammogram before their first contact with the study physician. To assess whether omissions in care were due to the provision of mammography screening by other physicians, we also determined the overall rate of screening for each study physician's practice, irrespective of the referring physician.

For each practice subpopulation (all outpatients, elderly patients, and women aged 50-69 years), information was first retrieved for each patient and then aggregated to produce summary measures for the practice (available from the authors on request). Variables used to summarize the practice population included age, sex, average income, education, geographic access to health care, comorbidity, prior health care use, and outcome-specific utilization of health services. Geographic access was treated as a potential confounder in comparisons of specialty consultation (distance to an urban center) and mammography screening rates (distance to a mammography screening facility) and practice population characteristics and outcome-specific utilization behavior as confounders for consultation, mammography, and symptom-relief prescribing rates. For inappropriate prescribing, no confounders were considered relevant because the decision to prescribe a relatively contraindicated medication is rarely justified by patient characteristics.

Multiple linear regression was used to test hypothesized associations between examination subscores and practice outcomes when adjusted for potential confounders. To account for differences in the precision of the estimated outcome rates for each physician, observations were weighted by the square root of the number of eligible patients in each physician's practice. The residuals were examined to assess whether a linear model was appropriate. All practice population characteristics were included in the regression models for consultation rate, symptom-relief prescribing rate compared with disease-specific prescribing rate, and mammography screening rate to control for potential confounding due to differences in the composition of the practice population. Because physicians entered practice at different points in the 18-month period, we controlled for varying amounts of follow-up by including months of active practice as a continuous covariate in the regression models. Each examination score was modeled separately to avoid problems of multicollinearity in the estimation of the regression coefficients. The population impact of variation in examination scores was determined based on the differences in the rate of each outcome among patients seen by physicians at the upper (≥2 SDs above the mean) and lower (≥2 SDs below the mean) end of the score distribution. We calculated the difference in the number of outcomes that would be expected among patients seen by physicians with lowest scores if they experienced the same outcome rate as patients of physicians with the highest scores.

A total of 810 family medicine residents took the licensing examination in 1991, 1992, or 1993, and 614 were eligible for inclusion. Among the ineligible residents, 21 failed, 46 did not apply for Québec license, 17 had not started practice within the first 18 months, and 112 entered salaried practice. Of the 614 physicians included in the analysis, 56.0% were female, 85.8% were graduates from Québec medical schools, and 6.5% were foreign medical graduates. Most (90%) had graduated between 1989 and 1991; 36.5% took the examination in 1991, 36.6% in 1992, and 26.9% in 1993.

Average standardized examination scores for the 614 physicians were slightly below the mean of 0 for the reference group (Table 1). This is because the reference group had, on average, higher scores than the entire group of physicians who passed the examination, and they were relatively homogeneous in their performance. The percentage of study physicians who were 3 or more SDs below the reference group mean varied from 0.8% (OSCE) to 3.1% (prevention). For prevention, the average score for the reference group varied from 74% to 80% in different administrations (SD, 7.5%-7.6%). Thus, physicians who obtained the passing score of 50% for prevention would have a standardized score of –2.8 to –3.9 depending on the year in which they took the examination. Score reliability varied from 0.27 to 0.72. Correlations among examination scores varied from a low of r=0.36 for prevention and OSCE scores to a high of r=0.63 for diagnosis and management scores.

During the 18-month follow-up period, the 614 study physicians saw a total of 1116389 individuals, 16% of the total population in the province. Seventeen percent of individuals were seen by more than 1 study physician. Over 74 million claims for medical services and prescriptions were retrieved for these patients, 31574888 from the baseline period and 42850337 from the follow-up period. Physicians practiced in an average of 3.6 settings (median, 4; range, 1-7) in the first 18 months and the distribution by geographic location was 71.8% in urban or periurban areas, 25.4% in rural and remote settings, and 2.8% in mixed locations. The average outpatient practice population size was 2250 patients per study physician, of which 251 were eligible elderly patients and 161 were eligible women aged 50 to 69 years. The average (SD) length of follow-up was 15.8 (3.2) months.

During the follow-up period, the average consultation rate in the practices of study physicians was 71 per 1000 patients (SD, 51). On average, 108 elderly patients per 1000 (SD, 109; range, 0-563) were prescribed disease-specific medication and 126 elderly patients per 1000 (SD, 103) were prescribed symptom-relief medication, the most common drug being a benzodiazepine (54/1000). The average difference in prescribing rates was negative (mean [SD], −18 [68]/1000; range, −167 to 188), indicating that physicians prescribed symptom-relief drugs to proportionately more elderly patients than they prescribed disease-specific medications. However, the range indicates that differences in disease-specific and symptom-relief prescribing rates varied considerably among physicians. Seventy-one percent of study physicians prescribed an inappropriate drug to at least 1 of their elderly patients. The average rate of inappropriate prescribing across practices was 20 per 1000 elderly patients (SD, 31; range, 0-310). Approximately half (9/1000) of all patients who were prescribed an inappropriate medication were started on the drug therapy by the study physician, the most common drug being a long-acting benzodiazepine. The mammography screening rate was 51 per 1000 women aged 50 to 69 years (SD, 81; range, 0-750) when restricted to mammograms requested by the study physician and 218 per 1000 when all screening mammograms ordered for study physicians' patients were considered.

Higher consultation rates were associated with higher overall examination scores (Table 2). For every SD increase in a physician's overall competence score, 3.8 additional patients per 1000 would be referred for consultation.

Physicians with higher diagnostic and management scores were more likely to have higher rates of disease-specific prescribing relative to symptom-relief prescribing and were less likely to prescribe symptom-relief medication or inappropriate medication. With every SD increase in diagnostic score, 3.7 more elderly patients per 1000 would be prescribed disease-specific medication relative to symptom-relief medication, 6.5 fewer elderly patients per 1000 would be prescribed symptom-relief medication and 2.7 fewer elderly patients per 1000 would be prescribed potentially inappropriate medication. Higher management scores were not significantly associated with symptom-relief prescribing but were associated with higher relative rates of disease-specific to symptom-relief prescribing and lower rates of potentially inappropriate prescribing. The regression coefficients were similar when the analysis was limited to treatment that was started by the physician. To determine if the exclusion of salaried physicians biased our estimates, we evaluated the rate of inappropriate prescribing among elderly patients prescribed a drug by the study physician. Diagnostic score was the most significant predictor in models that included and excluded salaried physicians and the estimated regression coefficient was the same in both models (β, −5/1000; P =.01).

Higher rates of mammography screening were associated with higher scores in prevention and clinical assessment. For every SD increase in these scores, we estimated that an additional 5.4 to 6.6 women per 1000 would be screened for breast cancer. The inclusion of mammograms ordered by other physicians tended to lower the estimated regression coefficients for each of the examination scores.

Although a relatively small number of physicians obtained subscores that were 2 or more SDs below (7.2%-9.9%) or above (4.7%-6.5%) the mean, these physicians saw a considerable number of patients in the initial practice period. If patients of physicians with the lowest scores experienced the same rates of consultation, prescribing, and screening as patients of physicians with the highest scores, an additional 3027 of the 125383 outpatients seen by the lowest-scoring physicians would have been referred. Of the 9718 elderly patients seen, 179 fewer would have been prescribed symptom-relief medication, 912 more would have been prescribed disease-specific medication, and 189 fewer would have received inappropriate medication. For the 5335 women aged 50 to 69 years who were seen, 121 more would have received mammography screening.

In this study, we found that clinical competence, assessed by a combined licensing and certifying examination at the end of training, predicted consultation, prescribing, and mammography screening in initial primary care practice. The strengths of this study were that we were able to study all physicians who entered fee-for-service practice, assess their clinical competence at the end of training with state-of-the-art methods of assessment, and capture information on all patients seen by these physicians in the initial practice period. Both the strengths and main limitations of the study relate to the use of secondary databases for outcome assessment. On the positive side, we were able to avoid biases related to self-selection, self-report, and poor documentation. On the negative side, we could not measure practice organization or many outcomes that would be of relevance in assessing the adequacy of primary care delivery such as episodic and chronic disease management and patient satisfaction. The relationship of examination scores to these outcomes remains unknown. As clinical information is limited, we were unable to determine the appropriateness of resource use or whether extenuating circumstances explained rates of mammography referral or inappropriate prescribing. Limited precision in classifying outcomes would have likely led to an underestimation of the strength of the relationship between examination scores and practice outcomes. Overestimation of the effect would have been possible only if physicians with lower scores were more likely to see patients who did not require referral; were more likely to have indications for benzodiazepines, NSAIDs, or relatively contraindicated therapy; and were less likely to comply with physician referral for mammography.

This study supports the validity of licensing examination as measures of clinical competence. This is good news for licensing and certifying bodies because these examinations have been used to judge whether an individual has sufficient knowledge, skills, and judgment to deliver safe and effective medical care, and yet there has been a paucity of information about whether these assumptions are correct.⁷⁴ Furthermore, this study supports the contention that specific competencies are directly related to the aspects of practice in which these abilities are assumed to be required. For example, knowledge of prevention was an important predictor of mammography referral rate. These findings legitimize the approach taken in this examination to require demonstration of a minimum level of proficiency in each of the major aspects of clinical competence: diagnosis, management, prevention, and clinical skills. It also raises questions about the more common practice of establishing 1 overall passing score since this may mask important weaknesses in specific domains.

The observation that 71% of newly licensed physicians prescribed potentially inappropriate medication to their elderly patients, and for 1 physician that 31% of his or her elderly patients were prescribed potentially inappropriate medication, raises the question of whether the passing standard was high enough. Methods of establishing the passing scores for examinations have not had empirical information that could be used to assess the risk of specific deficiencies in knowledge or ability on the quality of care that would be delivered to the population or consequences for health outcome. Either a specified proportion of examinees at the lower tail of the score distribution fail, regardless of their ability (norm-referenced approaches), or a minimum absolute standard is established that would be expected to fail about half of theoretically "borderline" candidates (criterion-referenced approaches). Practice outcome information such as that produced in this study could be incorporated into criterion-referenced methods of standard setting, and this may be an important area for future research.

From the perspective of training programs, there are 2 relevant issues. First, from the literature we know that the quality of a training program has been shown to be positively associated with the scores achieved on certification examinations,⁷⁵ and furthermore, that certification examination scores, at least in internal medicine, are associated with quality of practice as rated by colleagues.³⁷ This means that improvements in training programs may have an appreciable effect on the quality of care delivered by graduates. Second, scores on licensing examinations tend to be highly correlated with scores on the same types of tests taken during medical school.^76- 80 If licensing examination scores predict some aspects of future practice, then scores achieved during medical school may also be predictive. This supposition should be tested because undergraduate and postgraduate training programs have the opportunity to provide remedial programs for deficiencies identified at a much earlier point in training.

The focus in medical education and licensing examinations has usually been on the individual physician, not on the impact that a physician will have on his or her patient population. This study quantified the number of patients who are influenced by new physicians. Indeed, one sixth of the provincial population, more than a million patients, received services from these physicians in the first 1.5 years of practice. Even when risks are small, such as the case in this study in which only modest associations were evident between examination scores and practice outcomes, the effects of potentially suboptimal care delivery by lower-scoring physicians may be appreciable if a substantial proportion of the population is affected. If this effect persists over time, the population-attributable risk of receiving treatment from physicians with lower levels of competence may be considerable. This study could not answer this question. We do not know if the behavior exhibited in the early period of practice had an impact on health outcome or if these behaviors persist over time. Nevertheless, the associations we did observe raise some questions about current policy and practice.

First, we substantiated the observations of earlier studies^40- 41; physicians who have higher competency scores are more likely to refer their patients. What does this mean? We could assume that high-scoring physicians are overly fastidious in early practice. Alternately, at a time when health policy has encouraged a reduction in use of specialty consultation for cost containment, we could and should worry that physicians who have lower scores may be less likely to recognize the significance of a patient's problem or their own limitations. If this were the case, the apparent cost savings associated with low referral rates may mask poorer quality of care and even greater costs, both human and administrative, of avoidable morbidity.⁸¹

Prescribing activity of physicians has also come under the spotlight in health policy reforms and drug review assessments.^{13- 20,82- 84} Annual expenditures for prescription drugs have increased substantially,^85- 87 and drug-related illness is now claimed to be the sixth leading cause of mortality in the United States.⁸⁸ The obvious question for both consumers and payers is whether prescription drugs are being used wisely. This study showed that drugs that may be overused or relatively contraindicated in the elderly are more likely to be prescribed by physicians with lower scores in diagnosis and management. We have no way of knowing whether these prescriptions were justified, but our results supported the conviction of McGavock et al^46- 47 that high rates of symptom-relief prescribing compared with disease-specific prescribing may be more common in physicians with poorer skills in diagnosis and management. A considerable investment has been made in interventions to improve the drug-prescribing habits of practicing physicians.^31,56,89 This study provides new insights into this problem, namely, that training and licensure may provide a new avenue for preventive intervention that ultimately may be far less costly than remedial interventions for practicing physicians.⁹⁰

Physician recommendation has been identified as the most important determinant of the likelihood that a woman will receive a mammogram.^91- 94 Optimal levels of screening have not been achieved,^6,8- 10,95 and physician knowledge and attitudes have been identified as significant determinants of their intention to institute screening practices.^29- 30 To our knowledge, our study provided the first direct assessment of the association between competence in clinical assessment and prevention and mammography screening behavior. Our findings are important because physicians who are more likely to screen for breast cancer are also more likely to carry out other preventive services.^6,96 Changes in training programs and licensing examinations may provide a means of optimizing preventive care, and this should be the subject of future research.

Licensing examination scores are significantly associated with some aspects of initial practice behavior. Changes in the passing standard and training may provide an appropriate and effective method of optimizing practice. To determine if this is true, future studies should broaden the scope of practice activities assessed to include continuity of care, other aspects of preventive care and counseling, and the quality of episodic and chronic disease management. The association between examination scores and patient outcomes such as patient satisfaction, quality of life, and avoidable morbidity and mortality would also be important to assess. Future research should also determine if the association between examination scores and practice activity persist over time and whether examinations taken earlier in training are also predictive of future practice activity.