National Survey of Internal Medicine Residency Program Directors Regarding Problem Residents FREE

Since 1972, the American Board of Internal Medicine (ABIM) has relied on residency programs to evaluate the readiness of eligible candidates for certification. Accordingly, attempts have been made to standardize resident evaluation at the program level, and to identify those residents who fail to meet the evaluation norms. The ABIM defines a problem resident as "a trainee who demonstrates a significant enough problem that requires intervention by someone of authority, usually the program director or chief resident."¹ The problem is often manifested in 1 or more of the ABIM's 7 areas that relate to clinical competency: clinical judgment, medical knowledge, clinical skills, humanistic qualities, professional attitudes and behavior, medical care, and moral and ethical behavior. Residency program directors predictably have to work with problem residents and must consider the negative impact they may have on the residency program.

Although problem residents pose challenges for residency programs, there is little information on their epidemiology or management. A number of articles have addressed specific issues such as impairment due to emotional stress and substance abuse.^2- 25 To gain more understanding of the prevalence, identification, management, and prevention of problem residents, we conducted a national survey of internal medicine residency program directors.

Program directors of all internal medicine residency programs in the United States and Puerto Rico were targeted for inclusion in this study. A list of the 404 accredited internal medicine programs was obtained from the American Medical Association's Graduate Medical Education Directory.²⁶

For this study, we developed a detailed 89-item questionnaire focusing on the identification, prevalence, management, and prevention of problem residents. Relevant issues pertaining to problem residents were identified by reviewing the published literature using MEDLINE, PsychINFO, ERIC, and HealthSTAR and by conducting focused interviews with 6 internal medicine residency program directors in Maryland.

The questionnaire was organized into 7 main sections and used the ABIM's definition of a "problem resident."¹ The first section addressed the problematic behaviors and deficiencies noted among problem residents and their underlying causes. The second section explored how and by whom these residents were identified. The third section collected data about individual residency programs and the prevalence of problem residents within each program. The fourth section addressed the remedial actions used to improve the performance of problem residents. The fifth section addressed the disposition and outcome of problem residents. The sixth section asked about measures that program directors use to screen for potential problem residents during the residency application process. The final section collected data about each residency program and the program directors. The majority of the questionnaire items used a 7-point word-anchored Likert scale ("never, rarely, occasionally, half of the time, frequently, almost always, always").

After pilot testing the questionnaire with selected program directors and attending physicians, the confidential questionnaire was mailed to each program director in October 1999. Reminder postcards and e-mail messages were sent 4 weeks later, and a second mailing was sent to those who did not respond within 8 weeks. Finally, for those who still did not respond, follow-up telephone calls and faxes of the questionnaire were used to encourage full participation.

Survey responses were reviewed and entered into a computerized database.²⁷ The frequency distribution of responses was examined to find evidence of nonnormality, outliers, and other irregularities in the data. Descriptive statistics (mean, median, range, and SD) were used to summarize the responses to all questions. The sign test was used to compare various resident characteristics that may place an individual resident at increased risk of being identified as a problem resident. Spearman correlations were used to determine whether program characteristics (percentage of male residents, minorities, etc) influenced the risk of housestaff with the same characteristics being identified as problem residents. χ² Analyses were performed to detect differences in the characteristics, identification, and management of problem residents in community vs university teaching hospitals. Residency programs at Veterans Affairs and military hospitals constituted 5% of the sample and were not included in this subset analysis.

Of the 404 program directors in the mailing, 298 returned a completed questionnaire, for a 74% response rate. On average, program directors had been serving in that capacity for 7 (SD, 6) years. Eighty-four percent of respondents were male and 16% were female (vs 86% male, 14% female for nonrespondents; P = .49). Fifty-eight percent of respondents were from community-based teaching hospitals, 37% from university-based hospitals, and 5% from Veterans Affairs and military teaching hospitals (the corresponding percentages for nonrespondents were 59%, 39%, and 2%; P = .30).

The mean point prevalence of problem residents per program in the 1998-1999 academic year as reported by the program directors was 6.9% (SD, 5.7%; range, 0%-39%). Six percent of responding program directors stated that there were no problem residents in their programs. The majority (64%) stated that the prevalence had remained constant during their tenure as program directors. Twenty-one percent believed that the prevalence had decreased, which some respondents attributed to a better pool of applicants, an improved selection process, and the closer surveillance of residents with frequent feedback. Fifteen percent reported that the prevalence had increased; some of these respondents thought that this was due to a sense of entitlement by the recent cohort of residents and increased complexity of patient illness.

The most prevalent deficiencies identified by respondents were insufficient medical knowledge, poor clinical judgment, and inefficient use of time (Table 1). These were identified by 48%, 44%, and 44% of program directors, respectively, as being present "half of the time or more frequently" in problem residents. Situational, personal, or professional stresses and depression were believed to be the most frequently recognized underlying causes (Table 1).

Sixty-seven percent of program directors stated that the return rate of attending physicians' evaluation of residents was more than 75% in their programs. While the majority of program directors (74%) did not think that low evaluation return rate had been an issue in identifying problem residents, 60% strongly agreed or agreed that it had been difficult to convince problem residents of their deficiencies because of a lack of honest and accurate written evaluations from attending physicians. Table 2 lists the processes and personnel through which problem residents were identified. Eighty-two percent of program directors reported that direct observation in clinical setting was the most frequent method by which problem residents were identified. Critical incidents and poor performance at morning report or other conferences also frequently led to the identification of problem residents. Chief residents, attending physicians, and program directors most frequently discovered problem residents, although other residents and nursing staff also did so. Problem residents seldom identified themselves as such (Table 2).

Program directors were asked about the influence of certain personal characteristics of residents on the likelihood that a trainee would be identified as a problem resident. Most program directors did not believe that any of these characteristics had a bearing on this matter. Although these variables were thought to be predictive by some program directors, the proportion of program directors who perceived a given factor as decreasing a resident's risk of becoming a problem resident was always smaller than the proportion who perceived that factor to increase the risk (Table 3). Residents who were older than 35 years, international medical graduates (IMGs), and those of an underrepresented minority (this term was not defined in the survey instrument) were considered to be at greatest risk of being identified as a problem resident. Based on the perception of the program directors, these 3 factors were significantly more likely to be associated with being identified as a problem resident (P<.05 for each) compared with other characteristics, such as single marital status, male sex, or being a parent. However, the correlation test results showed that as the percentage of older residents, minorities, and IMGs increased within programs, program directors were less concerned about the potential for poor performance of these residents. Furthermore, IMGs were significantly less likely to be identified as problem residents in programs with high proportions of IMGs (P = .008).

Interventions that were reported by program directors as having helped to correct the identified problem, in situations in which the intervention was applicable, are listed in Table 4. The most frequent helpful intervention for problem residents was more frequent feedback sessions, followed by assigning a mentor for structured supervision, probation, psychiatric or psychological counseling, strict behavioral guidelines, and remedial didactic curricula.

When asked about their comfort level in dealing with problem residents, 88% of program directors were confident of their program's ability to identify them. Seventy-six percent were confident in their program's ability to diagnose or understand the causes of a problem resident's difficulties, and 58% were confident of their program's capability to manage problem residents.

In terms of outcomes seen in problem residents, program directors estimated that 57% finished their residency training at the appropriate time, 18% continued in the residency program but required additional training time, 9% transferred to another internal medicine residency program, 10% transferred to a different residency program (other than internal medicine), and 4% left the profession of medicine.

With respect to the legal repercussions of interacting with problem residents, 49% of program directors acknowledged threats of litigation as a result of their dealings with problem residents during their tenure as program directors. Fifteen percent of program directors indicated that their interactions with problem residents have resulted in actual lawsuits.

Program directors were asked to describe the measures in place that may benefit and support the well-being of their housestaff. More than 90% of the programs have the following: (1) orientation at the beginning of residency regarding expectations and supportive resources, (2) limited number of admissions on call days, (3) structured days off during ward rotations, (4) planned social events, and (5) parental leave policies. Seventy-three percent of programs have an advisory system, 48% sponsor retreats for interns and residents (ie, outings away from the hospital setting to assist in personal and professional development), and 35% offer intern/resident support groups or personal awareness groups.

Measures that program directors reported using to prevent deterioration and enhance the development of problem residents included regular meetings with program director and/or chief residents (94%), timely evaluations from attending physicians (87%), regular meetings of attending physicians with program director and/or department chairperson before and after ward rotations (49%), and switching rotations if the problem resident is particularly stressed and unable to perform at a high level (29%).

Program directors were questioned about the use of multiple factors in screening for potential problem residents during the application process. "Audition" rotations, interviews, board scores, and school records (eg, grade point average, ranking) were reported most frequently as being helpful (Table 5).

Subset analyses were performed to compare university- and community-based residency programs. The prevalence of problem residents was higher in community-based programs (8.1%) than in university-based programs (5.0%) (P<.001). Community-based residency programs had more residents who are male (P = .003), underrepresented minorities (P = .002), older than 35 years (P = .009), IMGs (P<.001), and parents (P<.001). Table 6 shows additional differences between university- and community-based residency programs on issues related to problem residents. Program directors from university-based hospitals identified depression as being more common among their problem residents. Program directors at community-based programs reported that they more frequently identified problem residents, while those at university-based programs reported that chief residents more frequently identified problem residents. Psychiatric or psychological counseling was reported as more frequently helping problem residents in university-based programs, perhaps reflecting the higher frequency of reported depression in these programs. In screening applicants, community-based programs used board scores as a predictor more frequently than did university-based programs.

Problem residents are moderately prevalent among internal medicine residency training programs. However, heightened awareness by medical educators may have resulted in an even higher estimate of the prevalence of problem residents. This may explain why our estimate was lower than that of the ABIM, which through on-site visits had estimated that 8% to 15% of residents have serious problems.²⁸ Many of our findings, such as identification, apparent and underlying causes, management, and preventive measures, represent new data.

Chief residents and program directors most often identify problem residents. The finding that problem residents do not generally identify themselves raises the question of whether there is accurate self-evaluation among these house officers. Other factors that might play a role are denial, embarrassment, fear of consequences, or a concern about the inability to improve in a problem area.

Program directors acknowledged that multiple apparent deficiencies and underlying causes frequently contribute to the suboptimal performance of problem residents. While the apparent deficiencies manifested by the problem residents are often noted first, underlying issues may be at the root of the problem. Heightened sensitivity to situational, emotional, cognitive, and cultural difficulties may lead to improvements in the apparent deficiencies.

Management of problem residents can be difficult, and litigation is a possible consequence.²⁹ Both threatened and actual lawsuits were reported by a surprisingly high number of respondents. The ABIM emphasizes the importance of documentation and due process; many residency programs can likely improve in this area.³⁰

Reliable predictors of individuals at risk for becoming problem residents could be used by programs in selecting candidates and targeting preventive strategies. "Audition rotations," interviews, board scores, and school records were reported as useful processes in screening for problem residents. Unfortunately, there is a lack of information on the efficacy of these variables or processes in predicting which applicants will become problem residents. It may be difficult to predict individual responses to the stresses of residency, and perhaps more emphasis should be placed on detecting problem residents early.

The majority of the program directors believed that personal factors do not affect the chance that an individual house officer will be identified as a problem resident. Among the program directors who did not share this view, residents who were older, underrepresented minorities, or IMGs were thought to be at greatest risk for becoming problem residents. These results raise the issue of whether medical students or housestaff with these characteristics may be discriminated against during the application process or training. Cultural differences, language difficulties experienced by some IMGs, expectations, and previous experiences may place these groups at a disadvantage at the beginning of residency training. Discrimination has been described in residency training programs, and program directors need to be cognizant of these findings.^31- 32

Several limitations of this study should be considered. First, the results are derived from a cross-sectional survey of program directors that relied on self-report, and therefore, causal relationships and efficacy of management strategies could not be assessed. Second, since program directors were asked to think about their experiences with problem residents during their tenure, recall bias may have occurred as program directors may preferentially remember specific cases or residents who were most challenging. Third, because of the sensitive nature of the content addressed in this confidential study, some respondents may have been overly cautious. Fourth, there are some areas, such as the prevalence of underlying factors, in which it would have been clearly preferable to ask the problem residents themselves. Finally, the results of this study may not be generalizable to training programs other than internal medicine.

Further research on problem residents may address issues such as the identification of reliable and valid predictors, trials of specific preventive and management strategies, and detailed assessments of the outcomes and disposition of problem residents. Investigations from the perspective of the problem residents themselves would be useful. Furthermore, among residency programs in which the prevalence of problem residents is significantly higher than the national mean, consideration of the system as the problem (as opposed to the residents) should be investigated.

In summary, our findings show that the vast majority of internal medicine residency programs have problem residents. The presenting characteristics and underlying issues are diverse, so identifying and understanding problem residents is a challenging task. An individualized approach to managing each problem resident may be most successful.