Preparedness for Clinical Practice:  Reports of Graduating Residents at Academic Health Centers FREE

The last comprehensive study measuring the preparedness of physicians early in their practicing careers, the 1991 Robert Wood Johnson Foundation's survey of young physicians, found that 80% of respondents thought their formal medical education did a "good" or "excellent" job of training them for clinical practice. However, many respondents felt unprepared for a variety of conditions they would encounter in their professional life, such as identifying depression, treating patients with severe disabilities, and treating elderly patients.¹ Subsequent specialty-specific studies have underlined these apparent gaps in physician readiness for practice. Physicians in pediatrics,² general preventive medicine,³ rural practice,⁴ and neurosurgery⁵ have been found to be underprepared for conditions and tasks for which residency ideally should have prepared them.

A number of observers have concluded that many physicians are also not prepared to provide services and manage conditions with particular relevance to underserved populations, such as dietary counseling,⁶ alcohol abuse,⁷ human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS),⁸ child abuse,⁹ or to care for patients with chronic conditions, such as asthma¹⁰ or complex conditions, such as cancer.¹¹ These deficiencies may seem surprising, given that nearly half of US physicians train at academic health centers (AHCs), which care disproportionately for underserved populations and patients with complex diseases.¹² While most studies show that focused short-term training can remedy some gaps in physicians' skills, such approaches seem inherently less desirable than providing adequate preparation as a part of routine training.

Questions have arisen as well about whether there exists a "residency-practice mismatch": a failure of graduate medical education to provide proper role models, supervised instruction, and diversity of patient experience that residents need for practice.¹³ There is concern that the current guidelines ensuring the adequacy of residency training, which focus on regulating the locus and content of residents' experience, may be insufficient and that these should be complemented with a competency-based system, as was implemented for neurosurgery at Johns Hopkins University.⁵

These findings and questions suggest the need for ongoing monitoring of residents' preparedness for practice at the end of their graduate medical education. We surveyed a national sample of residents in their last year of training in 1998 at AHCs (defined as medical schools and their closely affiliated or owned clinical facilities) in the United States. This survey provides benchmark data on self-reported preparedness of residents to undertake a wide variety of tasks common to their specialties.

We identified a stratified random sample of residents in 8 specialties (internal medicine, pediatrics, family practice, obstetrics/gynecology, psychiatry, general surgery, orthopedic surgery, and anesthesiology) who were scheduled to complete their training at AHCs in July 1998. For ease of presentation and because of space constraints, this article focuses on results for disciplines and problems relevant to adult patients. Findings concerning preparedness to care for children for pediatricians and family practitioners will be presented elsewhere.

We limited our study population to residents in their last year of training, so that we could elicit data about the entirety of respondents' graduate medical education experiences, including perceived preparedness for practice at the completion of residency. Because of resource limitations, we studied selected specialties chosen with advice from representatives (P. Griner, MD, and R. Meyer, MD, oral communication, February 1998) of the Association of American Medical Colleges. We studied residents in AHCs because these institutions often are at the forefront of educational reform and because data from the University HealthSystem Consortium (UHC) enabled us to assess the market competitiveness of local AHC environments.

We used a multistage process in compiling our sample of residents. First, we constructed a list of the major teaching hospitals most closely associated with US medical schools. We started by matching the 121 4-year US medical schools (excluding Puerto Rico and the armed forces) with the Association of American Medical Colleges list of integrated academic medical center hospitals. Integrated facilities are defined as hospitals that are owned by a college of medicine or those in which the majority of chiefs of service are also chairs of medical school departments. All integrated facilities were included in our sample. For medical schools with no integrated hospitals, we added hospitals identified as having a major affiliation¹⁴ with that school. In cases in which there was more than 1 hospital either affiliated or integrated with a given school, we selected the hospital with the most residents.¹⁵ Finally, to ensure that we captured other major teaching hospitals associated with AHCs, we compiled a list of the 100 hospitals with the largest numbers of residents, based on hospital reports to the Medicare program.¹⁵ We added all such hospitals if they were not already included on our list. The final hospital list included 162 US hospitals training 40 000 of the nation's 98 000 residents in 1998.¹⁶

In the second step of our sampling process, we identified all training programs at these facilities in our 8 selected specialties using the American Medical Association's Graduate Medical Education Database.¹⁶ These programs were then stratified by level of local market competition according to the classification of their parent hospital in the UHC staging algorithm. All AHC hospitals were assigned a market stage based on the UHC's market evolution model, which assessed the competitiveness of health care markets by metropolitan statistical area containing AHCs. Ratings ranged from least (stages 1 and 2) to most (stage 4) affected by competitive forces.¹⁷ Although the usefulness of the market evolution model has been questioned (1 study found that it failed to accurately predict the tendency of health care organizations to form integrated systems¹⁸), other analyses have found it to be a useful gauge of market competition in the 1990s¹⁹ and the resulting impact of competitive pressures on faculty attitudes, levels of AHC research productivity, and support for academic missions.^20- 23

In the final sampling step, we randomly selected programs within each market stage and specialty to achieve a target sample size of approximately 4800 subjects with 600 in each specialty and 1600 in each market stage. Because the numbers of stage 4 markets and thus AHC residency programs were fewer than in other stages, fewer than 1600 residents were included in the stage 4 stratum. Our final sample consisted of 4832 residents in 632 programs.

A total of 8 survey instruments (1 for each specialty) were designed, based on literature review, focus groups of residents, review of the relevant policies of the Accreditation Council on Graduate Medical Education, and comments from experts. The instruments were tested for understandability and accuracy of response (cognitive testing) by the Center for Survey Research at University of Massachusetts, Boston. Surveys were designed to take 15 minutes to complete.

The surveys were administered in the spring of 1998. All respondents received a mail survey administered by Datastat Inc, Ann Arbor, Mich. A sample of those failing to respond to the mailed instrument were contacted for telephone administration of the same survey performed by Atlantic Survey Research Inc, Boston, Mass. Response enhancement techniques included advance notification, multiple mailings, telephone follow-ups, and flexible scheduling. Furthermore, respondents were eligible to receive 1 of 3 cash prizes or in some cases payment for completed interviews. This protocol was approved by the Massachusetts General Hospital institutional review board. Our final response rate, adjusted for invalid sample (people who left their program or were assigned to the wrong specialty) was 65%.

Preparedness of Primary Care Residents to Counsel Patients. Residents in primary care or related disciplines (internal medicine, family practice, obstetrics/gynecology) were asked how prepared (very unprepared, somewhat unprepared, somewhat prepared, or very prepared) they felt to counsel patients about conditions (smoking, diet and exercise, HIV testing, domestic violence, substance abuse, depression, pain management, palliative care/end-of-life issues, and compliance with care issues), which other studies have found physicians to be underprepared to address.^{1,6- 7,11,24- 25} Responses of "somewhat prepared" and "very prepared" were considered "prepared."

Preparedness of Primary Care Residents to Care for Different Types of Patients. Residents in primary care and related disciplines were asked how prepared they felt to manage types of patients whom they might be asked to care for but for whom preparedness might be inadequate^1,26 (inpatients, ambulatory patients, critically ill patients, terminally ill patients, elderly patients, chronically ill patients, nursing home patients, HIV/AIDS patients, and substance abuse patients).

Preparedness of Residents to Diagnose and Treat Conditions or to Perform Tasks and Procedures Typically Associated With a Specialty. The residents in each specialty were asked how prepared they felt to manage a range of specialty-specific conditions or to perform services typically associated with their specialty. Conditions and services were chosen using input from focus groups and from clinical colleagues, including department chairpersons and training program directors who played a major role in graduate medical education.

In some analyses, scaled questions were collapsed into dichotomous variables (eg, prepared vs unprepared). Analyses were weighted to accurately represent national estimates, and to correct for nonresponse bias and for the probability of selection within a given stratum. We used SAS version 6.12 (SAS Institute Inc, Cary, NC) to construct the sample and to calculate descriptive sample statistics such as averages and proportions. Results of the analyses are presented as sample-weighted responses by category and may not always sum to 100% due to rounding. Because of their dual status as both surgical subspecialists and primary care providers for women, obstetrician/gynecologists are, as appropriate, grouped with primary care providers in some analyses and with surgical subspecialists in others.

Table 1 provides weighted and unweighted characteristics of our sample. On a weighted basis, 59% of respondents were male and 73% were graduates of US medical schools. Residents in internal medicine (34%) constituted the largest group of respondents by specialty with pediatricians (16%) and anesthesiologists (11%) ranking second and third.

Overall, 96% of primary care residents (internal medicine, family practice, and obstetrics/gynecology) were "very prepared" or "somewhat prepared" to counsel patients about smoking, 94% about HIV testing, 91% about diet and exercise, 89% about compliance with care issues, 87% about palliative/end-of-life issues, 85% about substance abuse and depression, 74% about pain management, and 67% about domestic violence. A total of 11% of internal medicine residents rated themselves as "very unprepared" to counsel patients about domestic violence (Table 2), 12% of family practice residents felt "very unprepared" or "somewhat unprepared" to counsel patients about compliance with care issues (Table 3), and 19% of obstetrics/gynecology residents felt "somewhat unprepared" or "very unprepared" to counsel patients about depression (Table 4).

Overall, 99% of all primary care residents felt prepared to treat inpatients, 94% to treat ambulatory patients or elderly patients, 91% to treat chronically ill patients, 90% to treat critically ill patients or terminally ill patients, 75% to treat substance abuse patients, 70% to treat HIV/AIDS patients, and 66% to treat nursing home patients.

More than 90% of internal medicine, family practice, and obstetrics/gynecology residents felt prepared to treat inpatients and ambulatory patients; more than 90% of internal medicine residents felt prepared to treat critically ill patients, chronically ill patients, and terminally ill patients; and more than 90% of residents in family practice felt prepared to treat elderly patients. Populations for which more than 15% of residents felt unprepared included nursing home patients, HIV/AIDS patients, and substance abuse patients (Table 2, Table 3, and Table 4).

Large majorities of residents reported high levels of readiness to undertake many common clinical tasks associated with their specialty but gaps in perceived preparedness were also evident.

Primary Care. More than 90% of all residents in internal medicine, family practice, and obstetrics/gynecology felt prepared to diagnose and treat diabetes, upper respiratory tract infection, and hypertension (Table 2, Table 3, and Table 4).

Psychiatry. More than 95% of psychiatry residents felt prepared to diagnose and treat schizophrenia, minor depression, major depression, suicidal tendencies, panic disorders, delirium, and obsessive-compulsive disorders, and to provide psychopharmacologic services. However, more than 10% felt unprepared to treat borderline personality or substance abuse, to provide short-term or long-term psychotherapy, to diagnose and treat patients with loss/bereavement issues, to diagnose and treat somatization, or to diagnose and treat eating disorders (Table 5).

Surgical Specialties. More than 90% of residents in obstetrics/gynecology felt prepared to perform cesarean deliveries, abdominal hysterectomies, and vaginal hysterectomies (Table 4); more than 90% of orthopedic surgery residents felt prepared to perform total knee replacements and total hip replacements (Table 6); and more than 90% of general surgery residents felt prepared to perform herniorrhaphies, appendectomies, total colectomies, femoral-popliteal bypass, and biliary tract surgeries (Table 7). More than 90% of all surgery residents felt prepared to communicate with referring physicians and to manage patients preoperatively and postoperatively (Table 4, Table 6, and Table 7).

However, like residents in primary care specialties and psychiatry, significant proportions of residents in surgical specialties reported feeling less than fully prepared to provide certain types of care. In orthopedic surgery 62% of residents rated themselves as prepared to perform spinal surgery and 54% to perform cancer surgery (Table 6). In general surgery, 81% reported themselves prepared to perform pancreatic surgery and 88% to repair abdominal aortic aneurysms (Table 7).

Anesthesia. More than 90% of anesthesiology residents felt prepared to administer spinal and epidural anesthesia, general anesthesia for patients with complex illnesses, anesthesia in day surgery, cardiac anesthesia, perform preanesthesia testing, manage acute pain, and administer postoperative intensive care (Table 8). However, 32% of anesthesiology residents felt unprepared to manage chronic pain.

Although limited by the use of residents' perceptions and by the cross-sectional nature of the survey, our data nevertheless provide some useful insights into the preparedness of residents completing graduate medical education. Residents tended to have a positive view of the overall quality of their training and overwhelming majorities were leaving their programs feeling somewhat or very prepared to manage most of the common clinical problems they are likely to encounter. These high levels of preparedness should provide reassurance about the overall quality of graduate medical education in the United States.

However, if residents' evaluations are any indication, their preparedness in nontraditional locations (eg, nursing homes) or for nontraditional patient populations (patients with substance abuse problems) may lag behind their preparation for care in traditional training environments. These findings suggest that AHCs now face the challenge of ensuring the quality of training for nontraditional educational experiences. Furthermore, in every specialty we studied, more than 1 in 10 residents in their last year of training felt unprepared to manage some clinical problems that they are likely to encounter in practice.

The implications of these reported gaps in preparedness are uncertain. Modern medicine is complex and diverse, and it is difficult to gain equal competence in all areas. Some residents go on to get additional training (especially in procedural specialties) that will prepare them better for some of the conditions, such as pain management among anesthesiologists, vascular surgery among general surgeons, or spinal surgery among orthopedists. Undoubtedly, some physicians will avoid treating problems they feel unprepared to manage. Nevertheless, the potential consequences of residents' perceived lack of preparedness in a number of important areas of practice deserves further assessment to evaluate implications for curricular design.

The discrepancy between our findings and the many recent anecdotal reports of the deterioration in medical education under the pressure of managed care and other stresses deserves further exploration. One possible explanation is that AHCs have succeeded in protecting educational missions through working both harder and smarter. If so, our data may testify to the ingenuity of AHC faculty and managers, but also raise questions about whether coping mechanisms can face additional pressures, such as the Balanced Budget Act of 1997. Alternatively, anecdotal reports may have overstated the problems facing the educational missions of AHCs. Only further research, including longitudinal studies of the content and quality of graduate medical education experiences, can shed light on these alternative explanations.

Our study had a number of limitations that should be mentioned. Perhaps most important is our reliance on reports of residents to assess their preparedness for practice. It is possible that self-perceived preparedness has little correlation with actual competency. However, others have shown that residents are as capable as their teachers at predicting their examination scores,²⁷ that students tend to underrate their preparedness relative to the assessments of their supervisors,^26,28 and that self-reported high levels of preparedness are correlated with good performance.²⁹ Additionally, self-reported preparedness has been used as an indicator of educational quality in other published studies.^1,30 Residents' reports at the end of training are not complete or definitive indicators of the quality of training but are clearly relevant. Student perceptions are widely used in education as an indicator of quality of educational experiences.

Overall, our data suggest that in 1998 residents finishing their training programs felt well satisfied with their preparedness for clinical practice. However, our data also suggest that gaps may still exist in the preparedness of physicians to manage the full range of patients, problems, and procedures they may confront as practitioners. These findings indicate the need for training programs to continue evaluating the appropriateness and diversity of experiences and instruction that residents encounter during their training.