Quality Assurance in Molecular Genetic Testing Laboratories FREE

Over the past decade many disease-causing or predisposing molecular changes in genes have been identified, permitting development of diagnostic tests. These analyses permit identification of affected persons, mutant gene carriers, and those at increased risk for developing selected disorders including single-gene defects and, recently, breast^1- 2 and colon³ cancer. Diagnostic tests with broader applications, such as assessing individual genetic response to pharmacologic agents,^4- 5 are expected to be available soon. However, concerns have been raised about performance of such testing. For example, it is widely accepted that integration of genetic testing into clinical practice requires preanalytical informed consent and preanalytical and postanalytical genetic counseling to prevent misuse of diagnostic results.^6- 8 Also, confidentiality policies must be developed to prevent genetic discrimination by insurers and others.^6- 9 Because molecular genetic testing frequently involves the use of reagents not approved by the Food and Drug Administration, is technical, and requires complex interpretation, it has been suggested that laboratories that offer such testing should be specifically regulated to ensure personnel standards and a high quality of testing. Currently, the Clinical Laboratory Improvement Amendments of 1988 (CLIA),¹⁰ mandates the regulation of clinical laboratories. They are based on a complexity model of testing with clinical molecular genetic analyses considered to be a high-complexity category. Although the high-complexity testing category includes 9 specialty quality assurance (QA) categories, a separate category for genetic testing does not exist. Recently, the National Institutes of Health–Department of Energy Task Force on Genetic Testing recommended that a specialty category of genetics be established for defining specific personnel standards for genetic testing and to monitor laboratory performance, including development of proficiency testing programs.⁶ However, members of the task force and others have cautioned that formulation of these regulations should not restrict access to testing.^6- 7 Thus, assessment of current clinical molecular genetic testing practices is important for rational development of public policy and regulatory requirements and to assess the impact of new regulations on QA and access, particularly for rare diseases.

We report herein the results of a survey of directors of molecular genetic testing laboratories in the United States which was designed to assess personnel qualifications, QA practices, and policies regarding informed consent, access to genetic counseling, and confidentiality.

CLIA¹¹ defines directors of high complexity testing laboratories as persons with overall responsibility for laboratory operation and for interpreting and reporting results to referring physicians or to patients. The study group of 746 potential directors included all 565 American Board of Medical Genetics certified clinical molecular geneticists, clinical biochemical–molecular geneticists, and clinical cytogeneticists, and all 109 noncertified directors listed in the Council of Regional Networks and HELIX¹² directories of genetic testing laboratories as of June 1997. An additional 72 directors were identified by contacting the chiefs of pathology at 118 medical schools and at their major affiliated hospitals (n=213) who provided names of persons directing clinical molecular genetic testing (response rates, 92% and 81%, respectively). To participate in the survey, laboratories were required to perform 1 or more of the following: molecular oncology testing, molecular genetic disease testing, parentage or forensic identity analysis, and/or fluorescent in situ hybridization (FISH) analyses.

A 20-multipart-question survey, approved by the Mount Sinai School of Medicine Institutional Review Board, was developed to collect data about laboratory setting; director, supervisor, and technician qualifications, training, and years of experience; type and number of testing services provided; QA practices regarding performance of Southern blot analysis, analytical gels, polymerase chain reaction, and FISH; types of specimens accepted and collection and delivery methods; information required on test requisition forms and reporting practices; licensing status; participation in proficiency testing (PT) program; employment of or affiliation with genetic counseling personnel; genetic counseling personnel qualifications and their roles in laboratory operations; policy on informed consent prior to testing and who obtains informed consent; and confidentiality policy. The survey was mailed in June 1997. Survey validation was achieved by (1) using previously validated items from surveys done by the Division of Laboratory Systems of the Public Health Practice Program Office of the Centers for Disease Control and Prevention to collect data about laboratory setting, qualifications and training of testing personnel, and specimen collection (25% of survey items); (2) conducting a test pilot with 15 molecular genetic testing laboratory directors to identify areas of clarification; and (3) analyzing 5 item pairs that would reflect internal inconsistency, which showed a low (<2%) level of internal inconsistency.

The QA practices of laboratories were assessed using the standards defined by the American College of Medical Genetics Laboratory Practice Committee.¹³ These guidelines establish minimum standards for each of the common methods used in clinical molecular genetic testing, including 7 standards for Southern blot analysis, 4 for analytical gels, 7 for polymerase chain reaction, and 5 for FISH (Table 1). A QA score was calculated based on responses to questions about standards for each method. A score of 1 was given for each met standard, and a score of 0 was given if the standard was not met. The polymerase chain reaction containment standard required that a minimum of 3 of the following containment procedures be in place to receive a point: dedicated pipettes, dedicated reagents and solutions, pipette tips that contain a filter, positive displacement pipettes, separation of preamplification and postamplification work areas, setup under a hood, and use of gloves. For FISH, the standards required a minimum of 5 metaphase cells for characterization of nonmosaic marker chromosomes or unknown regions in derivative chromosomes and a minimum of 10 metaphase cells for analysis of nonmosaic microdeletion/duplication. The total score of each laboratory was expressed as a percentage calculated by dividing the number of QA standards met by the total number of standards that would apply based on the director's reporting of which methods were used.

Dichotomous data were analyzed by forming a cross-classification of counts and using Pearson χ² statistic. Pairwise comparisons subsequent to the overall analysis followed the same pattern as described for analysis of variance. Generally, continuous outcomes were analyzed by analysis of variance, but if the data were markedly skewed (eg, number of tests performed per year), we used a nonparametric Kruskall-Wallis test and replaced the data with their ranking. Such a test is more sensitive in detecting differences in the median than the mean.

Responses given in open-ended questions were grouped to minimize the effect of outliers on comparison with QA score and to assist in interpretation of results. For the number of tests offered, data were grouped by whether the laboratory offered more or less than the mean number of tests. For the number of tests performed annually, respondents were separated into 4 groups: the lowest 10th percentile (1-30 tests), between the 10th percentile and median (31-225 tests), from median to the 90th percentile (225-1599 tests), and the upper 10th percentile (>1600 tests). All of the comparisons were prespecified with the exception of number of tests done annually.

The significance of differences in QA scores with respect to predictors (number of tests offered, number done annually, laboratory setting, PT program participation, CLIA certification status, and director degree and board certification) was evaluated using 1-way analysis of variance. If there were 3 groupings, unadjusted pairwise comparisons were performed if the overall test was significant (P<.05), based on the pooled error term. The P values were not adjusted to account for the number of possible comparisons, since the comparisons were preplanned. We used 4 groupings to analyze the number of tests performed and did not preplan comparisons. Since the comparisons in this case were suggested by the data, we used the Scheffe method of multiple comparisons, which is conservative and allows for all contrasts of interest.

Based on 2-tailed testing at the P=.05 level, we had 80% power to detect a 4% to 7% difference in QA scores. The difference we detected depended on the sample size in each group, reflected in the imbalance in the groupings. For example, we found a difference in the size of the CLIA and board certification groups of between a 9:1 and 12:1 ratio, which allowed power to detect a 7% difference. The imbalance was less for the professional setting for which we could detect a 6% difference. We found a 2:1 imbalance for the number of tests offered for which we could detect a 4% difference. For the comparison of directors with PhD and MD degrees we could detect a difference of 5%.

Of 746 potential laboratory directors contacted, 419 reported that they were not directing a molecular genetic testing laboratory. Of the remaining 327 individuals, 245 returned a completed survey (response rate, 75.9%). All laboratory directors responding to the survey had 1 or more doctoral degrees, and 94.7% were board certified. Of study subjects, 151 (62.0%) had a PhD degree, 52 (21.2%) had an MD degree, 40 (16.3%) had both an MD and PhD degree, and 1 (0.41%) had a DVM degree. Of those reporting board certifications, 189 (77.1%) were from the American Board of Medical Genetics (ABMG), 31 (12.7%) from the American Board of Pathology, 3 (1.2%) from the American Board of Clinical Chemistry, 2 (0.82%) from the Canadian College of Medical Genetics, and 7 (2.9%) from other boards. Of the 13 directors (5.3%) without board certification, all had MD degrees. The mean number of years directing a clinical laboratory was 18 (range, 1-35 years). Our follow-up with 82 directors who did not respond, verified that 72 were currently laboratory directors. Of these, 24% had MD and 59% had PhD degrees, not significantly different from responders (P=.61).

One hundred fifty-seven (64%) of the directors indicated that their laboratory was in a hospital setting, 44 (18%) were in a research setting, and 43 (18%) were independent or commercial laboratories. All responding research-based laboratories were functioning as clinical laboratories, reporting results to patients or to referring physicians; however, 24 of these laboratories (55%) offered testing only within an institutional review board–approved protocol. The setting could be determined for 72 of the 82 nonresponding directors; 44 (61%) were in a hospital, 15 (21%) were research laboratories, and 12 (17%) were independent laboratories, which is not significantly different from those who responded (P=.56). Of the 67% of laboratories employing a supervisor, 98% required a bachelor's degree or higher, and 64% required a master's degree or higher. The mean number of years' experience for supervisors was 13. Eighty-one percent of supervisors were certified by a specialty board or agency, 72% of laboratories required a bachelor's degree or higher for technicians, and 32% required professional certification. More than 97% of the laboratories trained technicians through the bench-training method (ie, other laboratory staff).

Eighty-seven percent of laboratories employed at least 1 ABMG-certified or eligible professional, and 83% reported an affiliation with 1 or more doctoral-level genetics professionals. Responsibilities of affiliated geneticists included laboratory direction (54%), clinical consultation to referring physicians (53%) or patients (47%), education of technical staff (35%), and test development (33%). Overall, 11% of laboratories reported that they did not employ and were not affiliated with doctoral-level genetics professionals.

Mean number of tests offered was 4.68 (range, 1-27). Mean number of tests performed annually was 872 (range, 10-31,500). Most laboratories (59%) performed less than 225 analyses annually, with 33% performing between 225 and 1600 analyses and only 8% with more than 1600 tests per year, or about 30 tests per week. There was no significant (P<.05) difference between laboratories in the 3 settings in the average number of tests performed in 1996. However, there was a tendency for highest-volume laboratories (>1600 tests per year) to be in an independent setting. For example, 8 (19%) of 43 independent laboratories processed more than 1600 tests vs 1 (2%) of 44 of research laboratories, and 11 (7%) of 157 hospital-based laboratories.

The total number of tests performed in each laboratory setting, by year, is shown in the Figure 1. The combined number of analyses was 97,518 in 1994, 134,230 in 1995, and 175,314 in 1996, corresponding to increases of 38% between 1994 and 1995 and 30% between 1995 and 1996. Of the laboratories, 70% offered direct mutation analysis for 57 disorders, 40% offered linkage analysis for 22 disorders, and 32% offered direct sequencing for 15 disorders. The most common analyses involving direct mutation detection were for fragile X syndrome, factor V Leiden, Prader Willi or Angelman syndrome, Y chromosome deletion, and cystic fibrosis in 45%, 29%, 27%, 26%, and 24% of laboratories, respectively. For linkage analysis the most commonly offered tests were for Duchenne muscular dystrophy and factor VIII deficiency in 11% and 9% of laboratories, respectively. Most of the least often offered tests (eg, for phenylketonuria and congenital adrenal hyperplasia) and tests offered by a single laboratory were being performed only in the research laboratory setting.

Fifteen percent of directors reported that their laboratory was using all 4 methods for which standards were assessed, 54% were using 3, 20% were using 2, and 9% were using 1. The latter group was entirely represented by laboratories doing FISH studies only. The mean total QA score for all laboratories was 90% (range, 54%-100%). Factors associated with a higher QA score included a test menu of more than 4 tests (P=.01), performance of more than 30 analyses annually (P=.01), director with a PhD vs an MD degree (P=.002), directors with board certification (P=.03), testing performed in independent (P<.001) or hospital settings (P=.01) vs research setting, participation in a PT program (P<.001), and CLIA certification (P=.006) (Table 2). Laboratory supervisor qualifications were not significantly associated with the total QA score (P=.24). The QA score for the 4 methods showed a mean of 89.3% for Southern blot analysis, 90.9% for analytical gels, 92.4% for polymerase chain reaction and 74.1% for FISH. Further analysis of the mean FISH score showed that a number of laboratories (16.9%) failed to meet standards regarding the number of cells analyzed.

Eighty-eight percent of directors reported use of a test requisition form. Of these, 74% request family history and 44% request a pedigree. Fifteen types of specimens were accepted by laboratories including whole blood (97.6%), cultured amniocytes (66.7%), postmortem tissues or fluids (61%), cultured fibroblasts (64%), chorionic villus cells (56%), bone marrow (54%), amniotic fluid (52%), and fetal blood (48%). Ninety-eight percent of laboratories issued a report, with the laboratory director signing all reports. The 5 laboratories (2%) that did not issue a report were in a research setting. A summary of the method used to perform the analysis was included in 88% of reports, but only 78% provided detail about probes, primers, and restriction endonucleases used. Suggestions for further testing were included in 93% of reports.

Ninety percent of laboratories had CLIA certification, 70% were licensed in the state of location, and an additional 14% were licensed by another state, with New York being the most common for additional licensure. Eighty-eight percent of laboratories participated in 1 or more PT programs, with 98 laboratories (40%) in College of American Pathologists (CAP) Programs in Molecular Genetics and 97 laboratories (39%) in CAP Programs in Cytogenetics. The former group represents 127 (77%) of all enrollees in the CAP Molecular Genetics survey for 1997 (Eric Grahek, written communication, October 1998). Research laboratories were less likely to have CLIA certification (59%) than hospital laboratories (94%; P<.001) or independent settings (98%; P<.001). Similarly, research laboratories were less likely to participate in PT (59%) vs hospital laboratories (95%; P<.001) and independent (95%; P<.001) settings.

Most directors reported that they provide access to genetic counseling, either through employment of counseling personnel (27%) or affiliation with a clinical genetics counseling service (43%). We found no statistically significant difference in access to genetic counseling among settings, although independent laboratories were more likely to employ a counselor than affiliate with a counselor. The role of counselors did not vary between settings with the exception of counselors in the hospital-based setting who more likely (82%) provided information about abnormal test results directly to patients vs genetic counselors in independent and research settings who reported results to patients 62% and 68% of the time (P<.001).

Forty-five percent of laboratories required informed consent prior to testing for at least 1 analysis offered, and 69% of laboratories had a written confidentiality policy. Informed consent was required prior to diagnostic analysis for a genetic disorder by 58%, prenatal diagnosis by 51%, presymptomatic testing by 44%, and predisposition testing by 34%. However, only 6% of laboratories with a requirement for informed consent would reject a specimen unaccompanied by consent. Informed consent was more frequently required by research laboratories (58%) vs independent laboratories (33%; P=.01) and hospital laboratories (27%; P=.001). Informed consent was significantly more frequently required by laboratories that employ genetic counselors (62%) than laboratories with an affiliation with counselors (41%; P<.001), or those offering no access (29%; P<.001). A similar trend was noted for confidentiality policy, which was more common in laboratories that employ genetic counselors (80%) vs those that have an affiliation (70%), or those not offering access to counselors (63%). However, pairwise differences were not statistically significant. Confidentiality was significantly (P=.03) more frequently required in laboratories offering BRCA1 and BRCA2 testing (94%) than in those that did not (70%).

The results of this survey on molecular genetic testing provide the first detailed information about personnel qualifications, laboratory QA practices, and informed consent and confidentiality policies. In general, all clinical molecular genetic testing laboratory directors responding to the survey met the director personnel standard for high complexity testing as defined by CLIA. This standard requires that directors have an MD or DO degree with board certification in pathology, an MD or DO degree with 1 year of laboratory training and 2 years' experience, or a PhD with board certification.¹⁴ The National Institutes of Health–Department of Energy Task Force further recommended that all directors should have formal training in human or medical genetics, with certification by the ABMG or an equivalent organization.⁶ In our series, 78% of directors met the ABMG certification requirement, with an additional 13% certified by the American Board of Pathology. Thus, a number of laboratories may not be in compliance with the task force personnel standard recommendation without further training and certification of the director, a change in personnel, or grandfathering of existing directors. In support of the board certification requirement, it is noted that laboratories with non–board-certified directors had significantly lower QA scores vs laboratories with board-certified directors (Table 2), suggesting that board certification may be a reliable indicator of director qualifications.

Information about clinical laboratories offering molecular genetic testing, their directors, and analyses being performed has been limited. Because molecular genetics is not a separate specialty under CLIA, the number of such laboratories cannot be easily determined by inspection of data maintained on clinical laboratories by the Centers for Disease Control and Prevention and Health Care Financing Administration. Also, a number of laboratories providing clinical molecular genetic testing services are not CLIA certified (10%), most of which identify themselves as research laboratories, further confounding efforts to determine the number of such laboratories and identify those who direct them. Previously reported studies designed to assess the extent of molecular genetic testing in the United States identified a smaller number of directors (n=104¹⁵; n=136⁶) vs the 245 responses reported herein. Similarly, the number of participants in the CAP molecular pathology survey (127 in 1997 and 150 in 1998) is smaller than that reported herein. Thus, our findings were provided by the largest number of laboratory directors and document molecular genetic testing availability in the greatest number of laboratories reported to date. The data show large increases in number of analyses performed between 1994 and 1995 (38%) and 1995 and 1996 (30%), indicating widespread availability of molecular genetic testing and its rapid integration into clinical practice.

The assessment of QA practices revealed that a number of laboratories had QA scores that may reflect suboptimal laboratory practices. Since the American College of Medical Genetics guidelines define minimum practice standards, developed to ensure accurate test results and correct data interpretation, the expectation is that laboratories achieve a score of 100%. Among survey respondents, 36 laboratories (15%) had QA scores below 70%, suggestive of substandard laboratory practice and with the potential for adverse clinical outcomes due to incorrect diagnoses. For example, misdiagnosis of chromosomal abnormalities has been reported after FISH analysis when the appropriate number of cells were not analyzed and molecular probes were not validated.¹⁶ Similarly, diagnostic errors resulting from polymerase chain reaction contamination have occurred.^17- 18 That 33% of these lower-scoring laboratories were directed by non–board-certified directors and that 45% were in a research setting suggests that these 2 factors may be important in identifying laboratories with possible QA deficiencies. Also, research laboratories were less likely to be CLIA certified or to participate in PT programs. However, this may reflect the lack of appropriate PT for genetic testing for rare disorders that occurs more frequently in the research laboratory setting than in hospital or independent settings and the fact that such laboratories may not rely on third-party reimbursement, which may provide an imperative for seeking regulatory approval, such as CLIA and CAP certification. Nevertheless, it is important to inform research laboratory directors who offer clinical laboratory testing about their responsibilities of registering with CLIA, complying with mandated standards, and participating in voluntary PT programs or interlaboratory comparison programs. It is important to identify mechanisms for ensuring quality in research laboratories that will not be viewed as onerous by their directors because they provide testing for a number of rare disorders that is not currently available in other settings.

Examination of the practices of clinical molecular testing laboratories regarding genetic counselor availability, required informed consent, and confidentiality policies showed wide variability. This may reflect a lack of consensus about whose responsibility it is to provide these services and safeguards. That 70% of laboratories provide access to genetic counseling and 45% require informed consent prior to at least 1 type of genetic analysis suggests that a number of laboratories recognize that counseling and informed consent are integral to their laboratory operation and to provision of genetic services. However, the finding that informed consent was required by only 45% of laboratories prior to presymptomatic testing and 34% of laboratories prior to predisposition testing is troubling because these analyses may pose a substantial risk to patients in terms of genetic discrimination. Moreover, for many of these analyses, the clinical significance of both positive and negative results is not always clear and there may be limited or no medical management options available.

These results document widespread availability of clinical molecular genetic testing in a variety of laboratory settings. The data regarding QA practices of such laboratories suggest that specific improvements in laboratory practices are needed to ensure a high quality of service.