Evaluation of a Risk Assessment Questionnaire Used to Target Tuberculin Skin Testing in Children FREE

Surveillance strategies for the prevention of tuberculosis (TB) in children have alternated between universal screening and targeted screening based on epidemiological risk. Between 1985 and 1992, there was a reported recrudescence of TB that accounted for a 35% increase in the number of cases involving children.^1- 5 In response, universal Mantoux skin testing using purified protein derivative (PPD) of tuberculin of all children was advocated by several authorities, including the Centers for Disease Control and Prevention (CDC) and the American Academy of Pediatrics (AAP).^6- 8

A decline in the prevalence of TB in the mid-1990s coincided with published reports indicating that universal tuberculin screening of all children was a costly and inefficient use of limited health care resources.^9- 10 In February 1996, the AAP's Committee on Infectious Diseases issued updated guidelines that supported focusing tuberculin skin testing on children who are at increased risk of acquiring TB.¹¹ In July 1996, the New York City Department of Health (NYCDOH) also revised its TB screening policy to allow for targeted screening.¹² Both the AAP and NYCDOH identified similar epidemiological risk factors for TB infection in the pediatric population, including (1) close contact with an active case of TB; (2) birth in or travel to an endemic region; (3) close contact with high-risk adults, including those with prolonged incarceration, human immunodeficiency virus (HIV) infection, homelessness, and intravenous drug use; and (4) HIV infection. Recently, targeted tuberculin screening based on similar risk categories has been endorsed by a joint statement of the American Thoracic Society, the CDC, and the Infectious Diseases Society of America.¹³ Although the risk categories were based on current knowledge, no formal analysis of the validity of this approach has been reported. To our knowledge, no other risk assessment categories have been developed and validated.

When any strategy is adopted for targeted screening, an essential question emerges: Are the risk assessment questions used to identify the need for a test both sensitive and specific? The present study was designed to determine the sensitivity, specificity, and predictive validity of the NYCDOH risk assessment questions used to assess the need for a tuberculin skin test.

We prospectively studied a group of children who lived in the South Bronx, New York City, NY. In 1996, this area had a reported TB case rate of 38.6 per 100 000, almost 5 times the national average.¹² Between August 1996 and January 1998 we recruited a consecutive sample of children (aged 1-18 years) who were seen for annual health maintenance visits at the Montefiore Medical Group. All children seen for health maintenance visits during the study period were considered eligible. Exclusion criterion was a prior documented positive PPD result. This study was approved by the institutional review board of Montefiore Medical Center, and written informed consent was obtained from a parent prior to participation.

Physicians and nurses at the health center had an in-service on the application of the questionnaire and interpretation of PPD results. The NYCDOH guidelines were posted in all examination rooms. All children and their caretakers were asked the following risk assessment questions: (1) Has your child had any contact with a case of TB? (2) Was any household member including your child born in or has traveled to areas where TB is common (eg, Africa, Asia, Latin America, and the Caribbean)? (3) Does your child have regular (eg, daily) contact with adults at high risk for TB (ie, those who are HIV infected, homeless, incarcerated, and/or illicit drug users)? (4) Does your child have HIV infection? Any "yes" response was considered a positive risk factor.

All participants received Mantoux tuberculin skin testing with 0.1 mL of 5 Tuberculin Units of PPD applied on the volar aspect of the arm to produce a wheal of at least 6 mm. Patients were instructed to return in 48 hours for test results, except those tested on a Friday were asked to return in 72 hours. Patients who did not return in 48 hours were telephoned that day and asked to come in later in the day or by the next day (within 72 hours). Postcards were mailed to all nonadherent patients without telephones, instructing them to return for retesting. We maintained a log of the test date, patient's name, medical record number, telephone number, and test result(s).

Initial interpretation of PPD results was performed by trained nurses and documented in the logbook and medical record. Nurses were blinded to the results of the questionnaire. If a nurse detected an induration or was uncertain, a physician assessed the patient immediately. All positive results were confirmed by a physician and measured with a ruler calibrated in millimeters. Parental reports and self-assessment of results were not accepted.

Positive PPD skin test reactions were defined as indurations >10 mm, in accordance with the AAP's guidelines.¹⁰ Chest radiographs were obtained for all patients with positive results.

Data were entered and analyzed using SPSS (Chicago, Ill). A "yes" response to any question was considered sufficient to warrant PPD testing. We determined the sensitivity (the probability that children with a positive PPD result would be targeted for screening), the specificity (the probability that children without a positive PPD result would not be targeted for screening), the positive predictive value (the probability of a positive PPD result among children targeted for screening), and the negative predictive value (the probability of children not targeted for screening having a negative PPD result) for each risk assessment question and for different combinations of the questions.

We calculated the prevalence of positive PPD results and reviewed the results of chest radiographic studies. To calculate odds ratios (ORs) and compare risk factors for TB, we classified the children on the basis of test results into a positive PPD group and a negative PPD group. We assessed differences in TB risk factors between these groups. Odds ratios for various combinations of risk factors for TB were calculated by univariate and multivariate analyses.

After initial analyses, children older than 11 years were found to account for 75% of those with a positive skin test result and no identifiable risk factor. Hence, we determined the sensitivity, specificity, positive and negative predictive values, and OR of age older than 11 years as a specific risk factor and in combination with the results of the NYCDOH questionnaire.

During the study period, 3093 eligible children were seen at the study site. No parents refused participation. Ten children were excluded because of a prior positive PPD test result. Of the 3083 who received Mantoux skin testing, 163 (5.3%) were lost to follow-up, leaving 2920 (94.7%) children. Fifty-five percent of these children were Hispanic, 44% were African American, and 50% were female. Forty-four percent of the children had no health insurance while 47% were covered by Medicaid.

The NYCDOH questionnaire identified 413 children (14%) as having at least 1 risk factor for TB. None of the children were identified as being HIV infected; hence, this item was dropped from further analysis. Of the 413 children with risk factors, 23 (5.6%) had positive skin test results, while of the 2507 children without a risk factor, 4 (0.16%) had a positive result. Children with at least 1 risk factor for TB were 35 times more likely to have a positive skin test result than those with no identifiable risk. Overall, Mantoux results were positive in 27 children (0.9%). Of the 4 children with positive skin test results not identified by the questionnaire, 3 (75%) were older than 11 years. Results of chest radiographs showed that none of the 27 children with positive skin test results had evidence of active disease.

The sensitivity, specificity, positive predictive value, negative predictive value, and OR for each screening question as well as combinations of risk factors are presented in Table 1. The full NYCDOH questionnaire had a sensitivity of 85.2%, a specificity of 86.0%, and a negative predictive value of 99.8%. Including age older than 11 years as a risk factor increased the sensitivity to 96.3% but decreased the specificity to 61.0%.

This is the first report to demonstrate the validity of an instrument used in determining the need for a tuberculin skin test in children. The NYCDOH risk assessment strategy had a high sensitivity and negative predictive value in this cohort. Children with at least 1 identifiable risk factor were 35 times more likely to have a positive skin test result. Using this strategy, the vast majority of children in our sample would not have required a skin test, yet very few children with a positive skin test result would have been missed.

Most of the children not identified by the NYCDOH questionnaire but who later had a positive skin test result were older than 11 years. This may be because children in that age group, due to their increased mobility, are at higher risk of exposure to individuals outside of the immediate household. The NYCDOH questionnaire primarily focuses on household contacts. When we included this age group as a risk factor, the sensitivity of the questionnaire increased to 96.3% but the specificity dropped to 61.0%. In our sample, it would have meant testing an additional 723 children to pick up 3 positive skin test results. These circumstances may be acceptable to some clinicians. Obviously, the prevalence of reactive skin tests and TB in a given population should be considered in making such a decision.

Of note, 10 children with prior positive PPD results were excluded from this study. A retrospective review indicated that 9 of the 10 children would have been identified by the questionnaire. Therefore, the exclusion of children with prior positive tuberculin skin test results biased the results against the validity of the risk assessment questionnaire.

We did not make any attempts to verify the caretakers' answers to the risk assessment questions. It is possible that some of the responses may have been inaccurate. However, in the clinical setting, the truthfulness of these responses may not be as important as the caretakers' self-identification as belonging to a high-risk category. For the same reason, we accepted at face value a response of "no" to the questions. Additional information was obtained in some instances; for example, we inquired about the specific countries of travel.

The low positive predictive values calculated in this report reflect the prevalence rate (1%) of positive skin test results. In the clinical setting, however, the negative predictive values are more relevant for screening tests in which the vast majority of children do not have the disease. Positive predictive values are more critical with diagnostic tests such as the PPD. Universal PPD testing in low prevalence areas yields high false-positive results because of the direct relationship between positive predictive values and prevalence. The children in this study were drawn from a community with a high case rate of TB. Even in this setting, universal skin testing would have yielded high false-positive rates, based on our findings. While the screening questionnaire was validated in this setting, this approach should be examined in different settings with varying prevalence of TB.