Prematriculation Variables Associated With Suboptimal Outcomes for the 1994-1999 Cohort of US Medical School Matriculants FREE

The US physician workforce is not of sufficient size or diversity to meet the population's health care needs.^1- 5 Ongoing efforts to address these workforce needs include increasing both the numbers and demographic diversity of medical school matriculants.^1,3,6 The effect of these efforts on physician workforce size and diversity will depend in large part on the extent to which matriculants complete medical school and graduate in a timely manner, prepared for advancement through graduate medical education (GME). Since the introduction of the US Medical Licensing Examination (USMLE) sequence, increasing numbers of medical schools have included passing scores on the Step l and/or Step 2 Clinical Knowledge (CK) examinations as criteria for advancement and graduation.^7- 9

We identified demographic and prematriculation variables associated with outcomes for matriculants who entered medical school since the introduction of the USMLE sequence. At follow-up through March 2, 2009, matriculants who graduated and had first-attempt passing scores on the USMLE Step l and Step 2 CK comprised the optimal outcome group because they were most favorably positioned for advancement through GME. The 3 suboptimal outcome groups were composed of matriculants who graduated but did not have first-attempt passing scores on the USMLE Step l and/or Step 2 CK, matriculants who had withdrawn or were dismissed for academic reasons, and matriculants who had withdrawn or were dismissed for nonacademic reasons.

A database constructed for this study included individualized, deidentified records for all 1994-1999 matriculants enrolled in US medical schools accredited by the Liaison Committee on Medical Education. The initial matriculation year of 1994 was selected because the USMLE sequence was not fully introduced until that year,¹⁰ and 1999 as the latest year to allow a sufficient follow-up period for all matriculants in our study; 96% of all students graduate within 10 years of matriculation.¹¹

The Association of American Medical Colleges (AAMC) provided individualized, deidentified Student Record System data updated through March 2, 2009, for all 1994-1999 matriculants, including matriculation year, sex, race/ethnicity, Carnegie Classification for undergraduate degree−granting institution, last-status description for matriculants no longer in medical school (academic or nonacademic withdrawal or dismissal, or graduated), and year of last status. The AAMC also provided responses to selected items on the AAMC Matriculating Student Questionnaire (MSQ), which is administered annually to incoming students and completed voluntarily on an identifiable but confidential basis.¹² Overall, the 1994-1999 MSQ response rates ranged between 93.6% in 1994 and 96.6% in 1997 (David Matthew, PhD, AAMC senior research analyst, written personal communication, March 4, 2010).

The AAMC provided matriculants' most recent verbal reasoning, physical science, and biological science subscores on the revised Medical College Admission Test (MCAT) and first-attempt USMLE Step 1 and Step 2 CK results as 3-digit scores and pass or fail, which were released with permission from the National Board of Medical Examiners. Records for each student were linked using a unique AAMC-generated identification number. The institutional review board at Washington University School of Medicine (St Louis, Missouri) approved this study with a waiver of consent.

Student Record System demographic variables included matriculation year, last-status date, sex, and self-identified race/ethnicity as reported to the AAMC by matriculants from a list of options on the American Medical College Application Service questionnaire. Race/ethnicity was categorized as Asian or Pacific Islander, underrepresented minority in medicine (including black, Hispanic, and American Indian or Alaska Native), other or unknown (including matriculants who self-identified as other, as multiple races, or did not respond to this question), or white (reference group). Medical school duration was calculated as years elapsed from matriculation to last-status year.

Additional MSQ variables included age at matriculation and premedical debt, which was categorized as $50 000 or greater, $25 000 to $49 999, $10 000 to $24 999, and $100 to $9999, and no debt (reference group). Based on responses to the MSQ item, “type of degree program in which you are enrolled,” matriculants enrolled in MD-only degree programs were included; those enrolled in dual-degree programs were excluded. For the MSQ item, “indicate any programs you participated in to prepare for a career in medicine or science,” yes or no responses were analyzed for laboratory research apprenticeship for college students and summer academic enrichment program for college students.

Matriculants obtained undergraduate degrees from educational institutions that included 29 different Carnegie Classification categories.¹³ A 6-category variable was created for Carnegie Classification undergraduate degree–granting institutions including (1) research universities with very high research activity, which was the reference category; (2) other institutions (all other undergraduate institutions); (3) baccalaureate colleges (arts and sciences); (4) master's colleges and universities; (5) other research universities with high research activity and doctoral/research universities; and (6) not specified.

A composite MCAT score was computed as the sum of verbal reasoning, biological science, and physical science subscores. A 7-category variable was created for analysis including score not available (to include students without MCAT scores), scores of less than 18, 18 to 20, 21 to 23, 24 to 26, 27 to 29, and greater than 29, which was the reference category. Composite MCAT scores of less than 18 were combined into a single category to ensure sufficient numbers in that low-score category, and all scores greater than 29 were combined because scores in this group were similarly associated with lower likelihood of academic difficulty during medical school.^{14(p916,Figure 3f)}

Categorical variables were created for Step l and Step 2 CK status as of March 2, 2009 (first-attempt pass, first-attempt fail, and no examination record). A 4-category outcome variable was then created for all matriculants in the sample using the Student Record System variable for last-status description as of March 2, 2009, and these two 3-category Step 1 and Step 2 CK variables: (1) withdrawn or dismissed for academic reasons, (2) withdrawn or dismissed for nonacademic reasons, (3) graduated and did not have first-attempt passing scores recorded on Step 1 and/or Step 2 CK, and (4) graduated and did have first-attempt passing scores recorded on both Step 1 and Step 2 CK, which was the optimal outcome and reference category.

Descriptive statistics are reported for each independent variable and the dependent variable. We report adjusted odds ratios and 95% confidence intervals from separate multivariable logistic regression models, which identified variables independently associated with each suboptimal outcome group compared with the optimal outcome group. Predictor variables were entered into each model in 3 blocks (1) MCAT scores, (2) sociodemographic variables (sex, race/ethnicity, and age), and (3) premedical variables (participation in laboratory research apprenticeship during college, participation in summer academic enrichment program during college, Carnegie Classification category for undergraduate institution, and debt). Separate logistic regression models were run to examine the associations between suboptimal outcomes and the main effects of each variable of interest, as well as models that added the interaction between the categorical race/ethnicity and MCAT variables in a fourth block. All tests were performed using SPSS version 17.0.3 (SPSS Inc, Chicago, Illinois). Two-sided P values of less than .05 were considered significant.

Of the 97 445 matriculants in the 1994-1999 cohort, we excluded the 178 who were still in school as of March 2, 2009, 81 who were deceased, and an individual whose degree was revoked. Of the remaining 97 185 matriculants no longer in school, 91 929 completed the MSQ at least in part (94.3% of all 97 445). After excluding 5815 MSQ respondents not enrolled in MD-only degree programs, 86 114 eligible MSQ respondents in MD degree programs remained. The final sample included 84 018 matriculants in MD programs with data for all variables of interest (86.2% of all 97 445 matriculants and 97.6% of 86 114 matriculants from MD programs who were no longer in medical school).

Of the 84 018 matriculants, 81 237 had graduated (96.7%), 1049 were no longer in medical school for academic reasons (1.2%; 653 dismissed and 396 withdrew), and 1732 were no longer in medical school for nonacademic reasons (2.1%; 121 were dismissed, 12 withdrew for financial reasons, 105 withdrew for health reasons, and 1494 withdrew for other reasons). Of the 178 matriculants still in school, 40 of 141 who had taken Step 1 (28.4%) and 18 of 42 who had taken Step 2 CK (42.9%) failed on their first attempts; outcomes, by definition, are suboptimal for these matriculants.

Of the total, the MSQ respondents included 95.1% of women, 94.2% of men, 92.9% of underrepresented minorities, 94.4% of Asian/Pacific Islanders, 95.3% of whites, and 81.2% of unknown or other race/ethnicity. The mean (SD) MCAT score was 29.3 (4.5) for MSQ respondents compared with 28.9 (5.0) for nonrespondents (P < .001).

The samples' characteristics are grouped by outcome categories in Table 1. The mean (SD) MCAT scores differed by race/ethnicity (P < .001) and were lower among matriculants who were underrepresented minorities (24.3 [4.8]) than among matriculants who were white (29.9 [3.7]), Asian/Pacific Islander (30.8 [3.8]), and other or unknown race/ethnicity (30.7 [4.2]).

The proportions of matriculants in each race/ethnicity group ranged between 1994 and 1999 (P < .001), from 15.8% to 19.9% for Asian/Pacific Islanders, from 13.9% to 15.7% for underrepresented minorities, and from 65.3% to 66.5% for whites. The mean (SD) MCAT scores increased from 28.5 (4.6) for matriculants in 1994 to 29.6 (4.3) in 1999 (P < .001). The mean (SD) scores increased from 211.1 (21.0) in 1994 to 216.2 (23.4) in 1999 for Step l (P < .001) and from 209.7 (22.9) in 1994 to 216.9 (22.7) in 1999 for Step 2 CK (P < .001).

Of the 82 090 Step 1 examinees, 4920 failed on their first attempt (6.0%). Of 81 275 Step 2 CK examinees, 3580 failed on their first attempt (4.4%). Of the 3580 examinees who failed Step 2 CK on their first attempt, 1313 also had failed Step 1 on their first attempt (36.7%). There were 1918 matriculants in the sample without USMLE records for Step l and/or Step 2 CK (2.3% of 84 018), including 12 in the group who graduated without first-attempt passing scores on Step l and/or Step 2 CK, 626 in the academic withdrawal or dismissal group, and 1280 in the nonacademic withdrawal or dismissal group.

Of matriculants in the sample with last-status date, 87% (72 145/82 971) arrived at final status within 4 years of matriculation, including 89.6% (65 909/73 526) of the group who graduated with first-attempt passing scores on Step l and Step 2 CK, 85.3% (1475/1729) of the nonacademic withdrawal or dismissal group, 77.2% (808/1046) of the academic withdrawal or dismissal group, and 59.3% (3953/6670) of the group who graduated without first-attempt passing scores on Step l and/or Step 2 CK.

The results of the 3 final multivariable logistic regression models of variables associated with each suboptimal outcome appear in Table 2. The goodness of fit of the partial models and the final models were all acceptable (Hosmer-Lemeshow statistic >.05). Matriculants were more likely to have suboptimal outcomes if they were Asian/Pacific Islander or underrepresented minority race/ethnicity; were older; obtained undergraduate degrees from institutions in Carnegie Classification categories other than research universities with very high research activity; had MCAT scores of 29 or less; had premedical debt of $10 000 or greater; or reported participation in a summer academic enrichment program during college. Matriculants were less likely to have suboptimal outcomes if they were women or reported participation in a laboratory research apprenticeship program during college.

In separate models, the effect of the interaction between MCAT scores and race/ethnicity was examined for each suboptimal outcome, but there was a significant interaction effect only for the suboptimal outcome of academic withdrawal or dismissal. In this model with the interaction effect, there was no change in the significance of any of the main effects of the predictor variables of interest after the interaction effect was entered. The only categorical comparison that was significant was for the group of underrepresented minority matriculants without MCAT scores (odds ratio, 4.48 [95% confidence interval, 1.43-14.01]; P = .01).

Suboptimal outcomes were observed for 11.3% of matriculants in our sample. Graduates with and without first-attempt passing scores on Step l and Step 2 CK were distinguished because graduates with first-attempt passing scores are more favorably positioned for entry into and progression through GME compared with graduates without first-attempt passing scores.

Program directors for GME place importance on Step 1 and Step 2 CK scores in resident selection.¹⁵ In the 2008 National Resident Matching Program (NRMP) Program Director Survey results, USMLE Step l score was the most frequently cited factor in selecting interviewees.¹⁶ Furthermore, among program directors who required applicants to submit Step 1 and/or Step 2 CK scores, 83.5% reported that they would seldom or never consider interviewing an applicant with a first-attempt Step l failure, and 87.7% reported that they would seldom or never consider interviewing an applicant with a first-attempt Step 2 CK failure.¹⁶ Applicants with first-attempt failing (or even lower passing) scores remain overrepresented among unmatched applicants in the National Resident Matching Program.^17- 18

Graduates with first-attempt failures on Step 1 and/or Step 2 CK also face challenges during GME. Many GME programs require USMLE sequence completion for contract renewal beyond the initial GME years.^19- 21 To do so, graduates must pass USMLE Step 3.²² Step l and Step 2 CK passing scores are prerequisites for Step 3 eligibility, and Step 3 scores correlate with MCAT, Step 1, and Step 2 CK scores.^22- 24 Thus, graduates without first-attempt passing scores on Step 1 and/or Step 2 CK are at risk for difficulty in timely USMLE sequence completion and are vulnerable to program dismissal. Many state licensing boards limit the number of attempts on each licensing examination and/or the time for USMLE sequence completion.²⁵ For these reasons, the optimal medical school outcome is graduation and first-attempt passing scores on both Step l and Step 2 CK.

The finding that lower MCAT scores were associated with an increased likelihood of suboptimal outcomes is consistent with other multi-institutional studies and a meta-analysis that documented positive associations between MCAT scores and each of Step l scores, third-year clerkships' grade point average, and Step 2 CK scores.^{14,24,26- 27} Underrepresented minority and Asian/Pacific Islander race/ethnicity was associated with a greater likelihood of academic withdrawal or dismissal and of graduation without first-attempt passing scores on Step l and Step 2 CK in a model that controlled for MCAT score; this is consistent with a report that nonwhite students performed more poorly in medical school compared with white students with the same MCAT scores.²⁶ Because these observations are from a model that also controlled for other variables, including premedical debt, further research seems warranted to identify additional variables amenable to interventions that may contribute to the disparate outcomes observed on the basis of race/ethnicity.

Lower MCAT scores did not preclude an optimal outcome for many matriculants. Because medical schools accept applicants with a wide range of MCAT scores, these findings may be of value in identifying matriculants who may benefit from additional support to maximize their likelihood of an optimal outcome.^14,28 The outcomes observed among matriculants without MCAT scores, and among the race × MCAT interaction group of underrepresented minority matriculants without MCAT scores, may be of interest to medical schools with special admissions programs that waive MCAT score requirements.

Women being at lower risk of academic withdrawal or dismissal differs from findings of an earlier study, which reported that women were at greater risk for academic difficulty.²⁹ As matriculation of women in medical school has reached parity with that of men, the physician workforce gender gap may continue to narrow.

Consistent with previous reports, older age at matriculation was associated with a greater likelihood of suboptimal outcomes.^27,29 Perhaps these matriculants might have had additional responsibilities (eg, family) during medical school or might have delayed their medical school entry to work to secure financing or take courses to strengthen their applications.³⁰

Almost 50% of matriculants in the sample received undergraduate degrees from research universities with very high research activity; graduates from other undergraduate institution categories were more likely to have a suboptimal outcome. These findings are consistent with the thesis that student experiences during their undergraduate years in university settings with very high research activity can promote success in the medical school environment.

Participation in a college research apprenticeship program was associated with a lower likelihood of suboptimal outcomes, but participation in a summer academic enrichment program during college was associated with a higher likelihood of suboptimal outcomes. Many summer academic enrichment programs are specifically intended for students interested in health professions careers who seek to strengthen their performance in premedical courses and on the MCAT, and so may be at greater risk for performance difficulties in medical school.

Because higher premedical debt was associated with greater likelihood of suboptimal outcomes, the low levels of socioeconomic diversity that exist among medical school matriculants may be even more pronounced among graduates.³¹ The findings regarding premedical debt and participation in college programs to prepare for a career in medicine, both of which are amenable to interventions, may be of particular interest to medical schools as they seek to meet the Liaison Committee on Medical Education accreditation standard on diversity.³²

More than 40% of matriculants who graduated without first-attempt passing scores on Step l and/or Step 2 CK were enrolled in medical school for more than 4 years. This likely reflects, at least in part, delays in advancement or graduation among matriculants enrolled at schools with passing score requirements for Step l and/or Step 2 CK prior to advancement or graduation. In 1994-1995, 87 schools had such Step l requirements and 53 schools had such Step 2 CK requirements for advancement or graduation⁷; in 2000-2001, 103 schools had Step l requirements and 72 schools had Step 2 CK requirements.⁸ In 2008-2009, 112 schools had Step l requirements and 93 had Step 2 CK requirements.⁹ Therefore, most contemporary matriculants who initially fail Step l and/or Step 2 CK are subject to delayed advancement or graduation if they eventually pass the examinations, or to withdrawal or dismissal if they do not. Those matriculants enrolled in one the relatively few schools without any such requirements may choose to take these examinations prior to or after graduation, or may not take them at all.

Despite trends toward increasing MCAT, Step l, and Step 2 CK scores, the proportion of matriculants in the optimal outcome group did not increase over time, which was likely due at least in part to changes in minimum passing scores on Step 1 and Step 2 CK. The initial Step 1 passing score of 176 in 1994¹⁰ was revised to 179 in 1998,³³ 182 in 2001,³⁴ 185 in 2007,³⁵ and 188 in 2010.³⁶ Similarly, the initial Step 2 CK passing score of 167 in 1994¹⁰ was revised to 170 in 1996,³⁷ 174 in 2000,³⁴ 182 in 2003,³⁸ and 184 in 2007.³⁹

Many matriculants in this study who withdrew or were dismissed from medical school had no USMLE records. School-specific curricula that facilitate identification and counseling of matriculants with difficulties before they attempt the USMLE sequence might be among the contributory factors.⁴⁰

This study of a nationally representative sample of medical school matriculants should be interpreted within the context of its limitations. Because we excluded matriculants who entered other types of medical degree programs, the findings can be generalized only to MD-degree program enrollees. The MSQ variables used in the study were by self-report, which may be prone to self-protection bias. Inclusion of only MSQ respondents may have introduced some selection bias because MSQ respondents had higher MCAT scores than nonrespondents. Matriculants' experiences during medical school and medical school–specific variables, such as cultural climate for demographically diverse student populations, curriculum, USMLE sequence policies, and quality of student support services would be expected to contribute substantially to attrition and graduation outcomes.^40- 43 Furthermore, because most matriculants in the category of nonacademic withdrawal or dismissal were not dismissed but had withdrawn from medical school for unspecified reasons, other unmeasured variables likely contributed particularly to this outcome. Because this is an observational study, causation cannot be inferred.

US medical schools accredited by the Liaison Committee on Medical Education are currently in a period of concerted efforts to increase enrollment and diversity of enrollees. These results regarding prematriculation variables associated with suboptimal medical school outcomes may help inform these endeavors.⁶