Measuring Trauma and Health Status in Refugees:  A Critical Review FREE

. . . in our country we don't have a thing called before war and after war. Since we are born we come to the war . . . that's how we are created. . . . Maybe the person who [made] those questionnaires, maybe he didn't know about our situation. Maybe he just knows . . . big combat or big wars. . . .—A Kurdish Man

A recent report by the US Committee for Refugees estimates there are 14.9 million refugees and 22 million internally displaced persons in the world.¹ Most have experienced significant trauma, including torture,^2- 4 as evidenced by prevalence studies in clinics and nonrepresentative community samples.^5- 12

Health problems of refugees have also been documented. Clinical research demonstrates a high prevalence of posttraumatic stress and depression symptoms,^6,10,13- 15 and community studies using self-rated scales^{2- 4,8,10,16} and structured diagnostic interviews^9,17- 19 have found wide variation in the prevalence of the symptoms of posttraumatic stress (4%-86%) and depression (5%-31%). Refugees experience multiple symptoms,^{4- 5,20- 26} perhaps due to the many types of insults experienced,^{4,6,20,23- 24,27- 28} yet the significance of these symptoms is not clear since many are not characteristic of posttraumatic stress disorder (PTSD), depression, or other defined disorders.^29- 35 A few community studies have shown that unexplained somatic symptoms are associated with low acculturation, high treatment seeking, psychiatric disorders, and self-identified medical problems, but these studies have not shown objective evidence of medical disorders.^3,16,36- 37 In addition to psychiatric morbidity, refugees have a high prevalence of dental, nutritional, infectious, and pediatric illness,³⁸ and they have greater self-rated impairment than the general population on the Medical Outcomes Survey SF (short form)-36 or SF-20, although neither survey has been proven valid or reliable in refugees.^4,8,39

Refugees clearly experience multiple stressful events that are associated with adverse health outcomes. Furthermore, they may have increased morbidity, decreased life expectancy, and a vulnerability to medical illness and poor health habits, as do other traumatized populations.^40- 44 However, data about refugee trauma and health status are often conflicting and difficult to interpret because various methods and instruments are used for data collection, analyses, and reporting.⁴⁵ Other methodological difficulties—such as translation and cultural differences,⁴⁶ and inadequate resources to fully assess symptoms—complicate accurate measurement.

Under the auspices of the New Mexico Refugee Project (NMRP), created to evaluate and improve the measurement of torture, trauma, and health status in refugees, a systematic literature review and 2 levels of assessment were conducted: (1) to assess the characteristics of the refugee trauma and health literature, and (2) to systematically evaluate instruments either developed in or adapted for refugee trauma and health research using 5 criteria.

Two primary sources were searched: (1) online databases using OVID from the inception date of each database through the first week of October 2001; MEDLINE from 1966, PsychInfo from 1967, Health and Psychosocial Instruments from 1985, CINAHL from 1982, and Cochrane Systematic Reviews from 1991, and (2) the NMRP database of 11 487 citations, developed using Endnote (ISI ResearchSoft, Carlsbad, Calif), a commercially available software package for literature archiving and searching.

The key terms used in the initial searches are shown in the . Combination operators were applied to the primary source citation results to identify English-language publications evaluating (1) refugee health status, (2) refugee trauma, and (3) measurement of trauma or health status in refugees. This method initially yielded 216 and 248 citations from the online and NMRP databases, respectively. Removal of duplicates within and between these databases left 181 and 135 publications, respectively (Figure 1). Two authors (M.H., N.L.) then reviewed all papers in the NMRP hardcopy files, which yielded an additional 78 publications, bringing the total citations found by search criteria to 394.

Data were extracted from the 394 articles (primarily by M.H.), with consensus obtained (between M.H. and N.L.) for 22 articles, using a form to record author; year of publication; whether the article was primarily about refugees, health status, and/or trauma; if it was empirical, review, or descriptive; if it evaluated statistical properties of instrument(s) (ie, reliability, validity, item analyses, factor analyses), and if so, which ones; and if it described the development or adaptation of instruments. "Adaptation of instrument" was defined as an instrument developed in nonrefugee research that was used in refugee research to adequately test at least one statistical property. The extracted data were entered into an Excel 2000 database (Microsoft Corp, Redmond, Wash) for analyses. Publications were excluded if they were reviews or were primarily theoretical or descriptive, were not primarily about refugee health or trauma, or were only about infectious diseases.

Instruments that were either developed or adapted and tested in refugee research were then evaluated according to 5 criteria described by Weathers et al⁴⁷: purpose, construct definition, design, developmental process, and reliability and validity. To meet these criteria, an article had to clearly state the purpose of the instrument, what construct was being defined and measured, how the instrument was designed and why, by what methods and with what rationale it was developed, and report at least one measure of validity and reliability. Others previously have reviewed general instruments for assessing trauma and health status^48- 49; our evaluation focused only on instruments either developed or adapted for use in refugee research.

Of the 394 publications identified by search criteria for further review, 187 were excluded for not meeting inclusion criteria (153 were not primarily about refugees and 34 were nonempirical or were not about trauma or health status measurement) and 24 publications had insufficient data for review (Figure 1). Thus, 183 articles were further analyzed. Of these, 178 (97%) were about health status (61% mental health, 16% physical health, and 20% both), and 82 (45%) about trauma. Ninety-one (49.7%) articles included quantitative data but did not evaluate measurement properties. Seventy-eight (42.6%) articles reported on the relationship of one measure to another, written with the assumption that validity was determined, even if the measures used had not shown validity. Fourteen articles (7.7%) were only about statistical properties of a measure (ie, reliability, validity, item analyses, factor analyses).

In the 183 relevant articles, 125 different instruments (ie, measurement tools such as questionnaires, surveys, and interview schedules) were described as being used to measure refugee trauma and/or health status. Only 12 instruments were developed specifically in a refugee sample and had sufficient detail in at least 1 article to allow for evaluation of the measure by the 5 criteria. Forty-one (22%) of the articles used 1 of these 12 instruments. Table 1 shows characteristics of these 12 instruments. Four of these 12 instruments measure health status (3 of which focused strictly on mental health), 3 measure trauma, 4 measure both trauma and health status, and 1 measures quality of care provided to refugees. None of these instruments fully met all 5 evaluation criteria. Three instruments completely met 4 criteria: the Harvard Trauma Questionnaire (HTQ),^7,50 Vietnamese Depression Scale (VDS),⁵⁸ and an unnamed scale developed by Bolton⁶⁰ that assesses mental health factors. One instrument met 3 criteria, 3 met 2 criteria, and 5 met only 1 of the criteria completely. None of these 12 instruments are published in the literature evaluated for this review, although components have been described and the full instruments may be available from the instrument developers or authors.

An additional 8 instruments developed in nonrefugee research that have been adapted for use in refugees were identified and evaluated (Table 2). Two of these instruments met all 5 criteria: Hopkins Symptom Checklist-25 (HSCL-25)^67- 69 and Beck Depression Inventory.⁸⁹ The other 6 instruments met 4 of the 5 criteria, but the purpose, construct definition, design, and development criteria were essentially met in Western, nonrefugee populations, and 6 of these instruments did not meet refugee-specific validity and/or reliability criteria.

Developed and Described Trauma Measures According to this search of the literature, 4 instruments that measure trauma have been developed in refugee research and are well described in the literature. The HTQ, developed by Mollica et al,^7,50 is a self-report questionnaire with 4 parts. The purpose of part 1 is to measure 17 war-related traumatic experiences. The scale was conceptualized by expert, consensus methods from clinical experience, and designed to allow respondents to check as many of 4 responses for each experience that apply to them ("did not happen," "experienced," "witnessed," or "heard about"). The HTQ manual describes its development, although it is not clear how items were chosen and designed.⁵⁰ A convenience sample of 91 Southeast Asian patients attending a psychiatric outpatient clinic were administered the HTQ, 30 of whom were readministered the test a week later. Excellent statistical properties were demonstrated: interrater reliability for all events (κ = 0.93); scale test-retest reliability (r = 0.89); and internal scale consistency (Cronbach α = .90).⁷ In a separate study of 30 Asian refugees, full-scale 1-week test-retest reliability was r = 0.62, but test-retest reliability for individual items ranged from poor (r = 0.23) to excellent (r = 0.90), meaning that some items were likely to be answered differently on the 2 tests while others were more stable.⁵¹ The HTQ, developed by a team with extensive refugee experience, was used in 22 of the 183 studies in this review. The trauma scale is reliable in clinical samples, although some items may not be reliable. It was rationally rather than empirically developed from clinical rather than community samples, and description about the construct and item development is scant. Thus, the 17-event list may be incomplete or biased, limiting generalizability. For example, experiences of women, as illustrated in the work by Allotey⁵² and Bonnerjea,⁵³ are not well represented. Furthermore, the design of multiple possible responses may confuse the respondent and limit reliability and therefore validity. Finally, validity and reliability of the torture item in particular have not been reported.

The 32-item Resettlement Stressor Scale (RSS), developed by Clarke et al⁵⁵ from their experience with Cambodian adolescents, was designed to measure stress due to resettlement. In one study with 38 adolescents, the RSS score discriminated between those who had psychiatric illness and those who did not using diagnostic interviews, and accounted for 11.7% of PTSD score variance but did not account for the depression score variance.⁵⁵ The War Trauma Scale (WTS), also developed by Clarke et al⁵⁵ from their clinical experience, consists of 42 items in both an interview and self-report format, measuring traumatic experiences inflicted by the Pol Pot regime. The WTS full-scale score had an adequate internal consistency (α = .74), acceptable interrater reliability (κ = 0.88), and accounted for 15.4% of PTSD score variance and 6.7% of depression score variance.⁵⁵ Both the RSS and the WTS demonstrated modest predictive validity of psychiatric disorder, and the WTS demonstrated acceptable reliability. They were developed by experienced investigators using rational rather than empirical methods. However, it is unclear from the literature how the items for each scale were constructed, developed, or designed, and they were reported in only 1 article.

The Post Migration Living Difficulties Scale (PMLD), developed by Silove et al,^9,54 is used to assess current life stressors of asylum seekers. Each of the 23 items of this administered survey is rated on a 5-point scale from "no problem" to "a very serious problem," with a composite score determined.^9,54 Its construct, development, and design are only partially described. Principal component analyses yielded 5 factors accounting for 69.8% of the variance of the 23 items: refugee determination process; health, welfare, and asylum problems; family concerns; general adaptation stressors; and social and cultural isolation.⁵⁴ These 5 factors were evaluated among asylum seekers, refugees, and immigrants. Asylum seekers scored higher than immigrants on all 5 factors, and higher than refugees on factors 1, 2, and 3. Refugees scored higher than immigrants on factors 2 and 3.⁵⁴ Thus, the PMLD is valid in discriminating between these 3 groups, but no other validity or reliability data are published. The PMLD is an important concept measuring life experiences other than war, but its usefulness is limited because of the lack of description about its design, development, reliability and validity, and scoring.

No refugee-specific instruments that assess prewar/conflict or nonwar/conflict trauma-related experiences in refugees were found. A number of general measures of lifetime trauma have been developed but have not been adapted or used with refugees.⁵⁶

Developed and Described Health Status Measures From our review, 2 instruments measuring health status that have been developed in refugee research are well described in the literature. Part 4 of the self-report HTQ, developed from clinical experience by Mollica et al,⁷ lists 30 symptom items, 16 generated from the Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition (DSM-III-R) criteria for PTSD, and 14 which are "presumably, culture-specific symptoms associated with PTSD." Possible responses are "not at all," "a little," "quite a bit," or "extremely." In the same convenience sample of 91 patients described earlier, internal consistency was excellent (α = .96), the symptom prevalence ranged from 44% to 92%, and the 1-week item test-retest reliability ranged from poor to excellent (r = 0.32-0.85; median, .59).⁷ An average item score of greater than 2.5 was predictive of a PTSD diagnosis by clinical interview (78% sensitive, 65% specific). The purpose, construct definition, and design of part 4 of the HTQ is clear. Modest reliability and fair validity in diagnosing PTSD was demonstrated in clinical populations. However, in a community study the sensitivity and specificity of the "greater than 2.5" cutoff score in diagnosing PTSD was 16% and 100%, respectively, and the most efficient score for diagnosis was 1.17 (sensitivity/specificity = 98%/100%).⁵⁷ The HTQ includes a limited range of possible symptoms, some are not reliable, and their ability to predict impairment has not been shown. Finally, as the authors discuss, generalizability of the HTQ and the construct validity of PTSD in general and in refugees need further study.

The VDS, a self-report questionnaire developed by Kinzie et al⁵⁸ to screen Vietnamese refugees for depression, was developed using a well described rational, consensus approach from extensive clinical experience. Culturally appropriate terms were added to existing Western symptoms of depression, and designed with items on a 3-point Likert scale. After pilot testing, the final 15-item scale measures 3 symptom types: physical symptoms associated with depression in the West, Western psychological symptoms of depression, and symptoms unrelated to Western concepts. The VDS is valid in discriminating between refugee patients with depression and those with anxiety or schizophrenia, and a cutoff score of 13 out of a possible 34 points demonstrated 91% sensitivity and 96% specificity for diagnosing DSM-III–defined major depression in a community sample.⁵⁸ Reliability has not been reported.

Less-Developed Health Status Measures Two additional refugee health status measures have been reported, but to our knowledge have not been formally named, used in other research, or published. Bolton⁵⁹ used 3 ethnographic qualitative methods to investigate Rwandans' perceptions of problems following the 1994 genocidal conflict and the local validity of Western concepts, and to adapt existing measures for local use. Two of the 18 identified problems about mental health were further developed. "Guhahamuka" (mental trauma), a concept that emerged after 1994, has 36 symptom items while "agahinda" (deep sadness or grief), an older concept, consists of 16 items. Guhahamuka and agahinda include all symptoms required for DSM-IV major depression and PTSD diagnoses, which was interpreted as supporting content validity of both syndromes in Rwandans. Agahinda was 95% sensitive and 38% specific for depression measured by the published cutoff point on the HSCL-25, and its test-retest reliability was modest but acceptable (r = 0.67).⁶⁰ This instrument is new and its design is not well described.

Beiser and Fleming⁶¹ used principal component analysis to identify 4 mental health factors (panic, depression, somatization, and well-being) in Southeast Asian refugees and Euro-Canadians from interviews using 6 existing scales as sources. The final 52-item administered instrument demonstrated acceptable internal consistency (α for the 4 scales ranged from .72-.91) and validity by discriminating between the 23 psychiatrically ill and the 30 well respondents.⁵⁹ Refugees and Canadians scored similarly on all 4 scales. The design and development of this instrument is not well described and reliability data were not reported.

Underdeveloped Potential Instruments

Five authors report work without fully discussing the construction of the actual instrument (eg, type or number of items, scoring). We include these works because the concepts and methods hold promise for refugee research. Ekblad et al⁶² used a 7-question qualitative interview to define and compare quality of life (QOL) between 14 Iranian refugees and 8 Swedes at a primary health care clinic in Sweden. The authors found that 3 of their thematic domains parallel 3 of 6 domains of the World Health Organization quality of life instrument (social relationships, level of independence, environment), demonstrating a form of validity.⁶³ Iranian refugees endorsed more social concepts of quality of life than Swedes, demonstrating discriminant validity. While this measure is incomplete, it is important since QOL is understudied in refugees⁴⁵ and is an important component of overall health and welfare.⁶³ Further development of QOL measures in refugees is needed.

Weine et al⁶⁴ report on an interview to investigate important concepts of provider (primary care professionals, social service workers, or refugee mental health professionals) knowledge, as well as attitudes toward, and service provision patterns for, Bosnian refugees with PTSD. The instrument was developed by the authors using rational, consensus methods. In their study of 30 randomly selected providers, primary care professionals had less knowledge about and provided less service to refugees with PTSD than did mental health or social service workers, demonstrating a form of discriminant validity.⁶⁴ No further design, development, or metric properties were reported.

Van Velsen et al¹⁴ report the development of the Survivor of Torture Assessment Record (STAR), a semistructured clinical interview that incorporates many instruments—such as the HSCL-25—and other investigator-chosen items to determine 3 scaled scores: trauma (scored 0-7), loss of health (scored 0-9), and social losses (scored 0-6). Validity was shown by the correlation between the trauma and loss-of-health scales (r = 0.59), and by the fact that all 3 scales distinguished between depressed and nondepressed refugees in London. Development was not described further and to our knowledge reliability has not been reported. McCloskey et al⁶⁵ used similar methods to integrate DSM-III-R criteria for PTSD, the Child Behavior Checklist,⁶⁶ 11 items about political violence, and 3 items about family conflict into an evaluation of trauma and health status for Mexican and Central American women and their children.⁶⁵ The authors report quantitative data but no statistical testing of these measures. While there are significant limitations to Van Velsen's and McCloskey's work, each demonstrate the integration of quantitative measures into a qualitative clinical interview, which can be used to enhance the validity of measures.

Cunningham and Cunningham⁵ developed 3 checklists to gather data about symptoms, trauma, and resettlement problems from case records at a treatment program for multinational refugees in Australia. The checklists were developed from literature about symptoms and trauma in refugees, and from the authors' experience with resettlement problems. Principal component analyses yielded 6 trauma factors and 6 symptom factors, with 2 trauma items accounting for 43% of the PTSD score variance. This research demonstrates the concept of using factor analyses to define potentially relevant trauma and health constructs for refugees, but the checklists are limited by not being empirically developed or tested for their statistical properties.

We found 8 instruments developed in nonrefugee research that had at least 1 statistical property tested in refugee research (Table 2). Two instruments, the HSCL-25 and the Beck Depression Inventory, met all 5 evaluation criteria.

The HSCL-25,^67- 69 a self-administered questionnaire originally designed to measure change in 15 anxiety and 10 depression symptoms in psychotherapy,⁷⁰ has been validated in the general US population⁶⁸ and used in many refugee studies. The content and design on a 4-point severity scale is acceptable to Indochinese populations, and reviews in the cultural psychiatry literature consider the measure valid.^71- 72 An average-item score greater than 1.75 indicates "clinically significant distress."⁷ Mollica et al⁶⁹ tested the HSCL-25 in 3 Indochinese groups, showing excellent test-retest reliability (r = 0.89 for total scale; r = 0.82 for each scale), good validity in predicting diagnosed depression (88% sensitivity, 73% specificity) or the presence of any major DSM-III-R–defined Axis I disorder with either scale or the total score (93% sensitive, 76% specific). The greater than 1.75 average-item score used as a diagnostic proxy for anxiety and depression is consistent with community data in general US populations.^67- 68 The HSCL-25 has good reliability and validity in clinical refugee samples, but is limited to symptoms of anxiety and depression, may not be a valid indicator of the full range of symptoms in refugees, and its ability to predict impairment has not been well studied in refugees.

The Impact of Events Scale (IES)⁷³ has been used in 8 of the 183 studies in this review. The 15-item measure has 7 intrusion and 8 avoidance items on 3-point descriptive scales measuring intrusive thoughts and body sensations and avoidance behaviors after trauma. It is valid and reliable in general populations,⁷⁴ and its development is well described. The 2 scales had satisfactory internal consistency (α = .82 and .74, respectively) and accounted for 41% of the variance in IES scores in a study of 1787 Croatian and Bosnian children,⁷⁵ confirming the 2-scale construct, although individual items fit differently than in the original 20-item version of the scales. Principal component analysis suggests a third scale, named "numbing," which requires further validation.^75- 76 Higher intrusion and total scores in children with more trauma events demonstrated validity, although trauma events did not predict avoidance scores. The IES scores also distinguish between 3 groups of adult refugees who have experienced torture, nontorture trauma, and migrants who have not experienced war trauma.¹⁰ Neither scale nor item test-retest reliability was reported.

The Symptom Checklist-90 (SCL-90),^77- 78 developed to measure change in psychological symptoms with treatment, consists of 10 scales and has been used in 4 of the 183 studies in this review. The SCL-90 depression scale was valid in differentiating depressed from nondepressed Hmong refugees who were either patients in a psychiatric clinic or who were from a community sample, and the depression scale correlated well with the Zung Depression Scale (r = 0.67), demonstrating concurrent validity.^79- 80 Further, the somatization scale correlated well (r = 0.40-0.52) with the somatic concern item of the Brief Psychiatric Rating Scale and the somatic anxiety subscale of the Hamilton Anxiety Scales.⁸¹ The depression scale of a translated Vietnamese version of the SCL-90 correlated well with the VDS (r = 0.81).³ However, we found no reliability testing of the SCL-90 in refugee research.

Other adapted instruments were tested in single studies identified in this review. The anxiety and depression scales from the Health Opinion Survey (HOS),⁸² an instrument derived from the general Cornell Medical Index to measure psychophysiological symptoms, were administered at time of interview to a community sample of 2180 Southeast Asian refugees from 3 countries. A factor analysis demonstrated that anxiety and depression were common and had the same meaning for all 3 groups.⁸³ Further analyses reported in a subsequent article demonstrated that a single factor resembling the concept of neurasthenia accounted for 40% of the distress scores on the HOS.⁸⁴ A community sample of Vietnamese refugees demonstrated high and persistent levels of physical and psychological symptoms on the Cornell Medical Index (CMI), a general health-status questionnaire, compared with normative data from the United States and Britain.^85- 87 Neither the HOS nor the CMI have had validity (vs a standard) or reliability tested in refugee samples. The Posttraumatic Symptom Scale-10, a 10-question survey measuring symptoms of PTSD, was found to have excellent internal consistency (α = .92) and test-retest reliability (r = 0.89) in Bosnian refugees, but has not been tested for validity in refugees.⁸⁸ The Beck Depression Inventory⁸⁹ demonstrated excellent internal consistency (α = .93) and excellent test-retest reliability (r = 0.92), and distinguished depressed vs nondepressed Hmong refugees against a clinician interview (94% sensitivity, 78% specificity), demonstrating validity.⁷⁹ The Norbeck Social Support Questionnaire (NSSQ), which measures dimensions of support that demonstrate excellent test-retest reliability and moderate concurrent validity in Western studies,⁹⁰ was adapted to study the relationship of 3 kinds of support (social network size, emotional support, esteem support) to health in Namibian refugees. The authors found that support and coping style moderated the relationship between chronic stress (years in exile) and health status (anxiety, physical symptoms, physical signs, and hospitalization in the previous year), demonstrating a form of predictive validity. ⁹¹ The NSSQ showed good internal consistency (α = .83), but no further adaptations or reliability testing have been conducted among refugees.

Half (n = 91) of the 183 articles about measurement of refugee trauma and health evaluated for this review reported quantitative data but did not report evaluation of association between or statistical properties of the instruments used. Forty-three percent (n = 78) reported associations between measures, assuming that validity had already been determined. In these 183 articles, 125 different instruments were used. However, only 12 instruments have been developed and tested specifically in refugee research; 3 of these met 4 of 5 evaluation criteria, but none fully met all 5 criteria recommended for a developed instrument, and none have been fully published in the literature evaluated in this review. Only 41 (22%) of the 183 articles used these 12 instruments. Another 8 well-described instruments adapted for use in refugee research were identified and evaluated; 2 of these instruments met all 5 criteria, 6 met 4 criteria (but these were all designed and developed in nonrefugee populations), and these 6 have not been tested for either validity or reliability among refugees. Of the 183 articles, 19% (35) used the 8 adapted instruments.

The primary limitations to accurate measurement of trauma and health status in refugees are the lack of theory-based construct definitions to guide the development and design of instruments specifically in refugee populations and inattention to use and reporting of sound measurement principles. These shortcomings may account for conflicting data between studies—different phenomena given the same name are being used to evaluate and compare populations. Improving measurement may help to clarify events that are traumatic and predictive of poor health, enable clinicians to better diagnose and care for patients, assist public health officials to develop better prevention models, allow scientists to conduct more useful research, and provide more accurate documentation of human rights abuses.

What instruments often lack is good, theory-based construct definitions that guide the design and development of measures, as shown in Table 1. For example, legal definitions that distinguish "refugee" from "asylee" from "internally displaced person" are not necessarily predictors of trauma experiences or health status.⁹² It might be that, for research purposes, a "refugee" or "displaced person" is best defined as a person who has fled his/her social living context because of threat to the safety or integrity of themselves or family members due to any cause (eg, war, civil conflict, disaster, oppression, or persecution that is explicitly or implicitly sanctioned by the state). Operationalizing this definition into a measure would focus on persons who have been displaced because of a threat to their safety. However, the optimal construct definition of "refugee" is an empirical question that requires further study.

Likewise, there is need for further study of what constitutes refugee trauma. No empirically developed instruments assess the complete range of trauma experiences in refugees. It is difficult to define all relevant events and types of events that influence health status,^{7,10,17- 18,20,95- 106} including the understudied effects of non–conflict-related events,^17,54 since trauma may precede and postdate experiences related to war and conflict, genocide, disaster, or oppression and because subjective experiences are highly variable.^{3- 4,19,21,54,93- 94} Thus, further community-based empirical research to better define the range and type of events that are associated with adverse health status is needed. Refugee researchers might consider adopting methodologies from life events research to better define how and what events are weighted as traumatic and predictive of poor health.^107- 108 The concept of "polytrauma" might be developed and used in research, since refugees experience multiple events in multiple contexts over time. This concept, applicable to other populations,¹⁰⁹ is especially important for refugee research to remind investigators of the multiple events to consider as moderators of health status.

Measurement constructs of health status are better developed than are those for trauma. Extant studies evaluate medical and psychological symptoms, disorders, diseases, and impairment. However, no community-based empirically developed instruments assess the full range of symptoms in refugees, and valid illness constructs associated with impairment are underdeveloped and require further study. For example, psychiatric symptom counts are less meaningful indicators of adverse health than if their relationship to impairment is established. Only a few community prevalence studies of psychiatric disorders using diagnostic instruments have been reported,^9,17- 19 and these did not assess the relationship of disorders to impairment. Symptoms, disorders, and even objective evidence of disease do not necessarily imply impairment, as demonstrated in people with renal disease, panic disorder, and heart disease.^110- 113 Illness, on the other hand, is defined by loss of functioning, and impairs a person in a highly contextual manner and is not necessarily defined in current medical nosology.¹¹⁴ Research has focused on infectious diseases, PTSD, anxiety, and depression, with some research focusing on physical injury, nutrition, and preventive health. Consideration must be given to other symptom complexes that are more strongly associated with impairment.^{7,20,35,97,115- 121} For example, it is not clear that "PTSD" is the most appropriate construct for traumatized refugees with symptoms currently defined as PTSD. Culture and language complicate diagnosis,¹²² and polytrauma is pathogenic for disorders that are different compared with those found in nonrefugee populations.^17- 18,123 The work of Bolton⁶⁰ is important in this regard as it demonstrates how community-based, empirical qualitative data validate refugees' illness experiences. However, this work has yet to demonstrate how local illness constructs are related to impairment. In addition to improving measurement about the full range of symptoms and valid illness constructs, measures of self-rated impairment for refugees are needed, since negative self-perceptions of health may predict future physical illness, mortality, and quality of life, independent of objective health status.^124- 127

Design and Development Instruments developed in community refugee populations using empirical approaches combining qualitative and quantitative methods may create measures that are more valid in representing the experiences of refugees than methods where data are only obtained rationally via expert and consensus approaches.^128- 129 Qualitative techniques, such as in-depth interviews and focus groups, help identify the range, depth, and meaning of possible responses in a population,^130- 133 and allow for development of culturally informed quantitative measures. These new instruments must then be validated using iterative statistical and field testing methods. Further, culturally informed quantitative instruments must be designed to be linguistically and visually acceptable and understandable to various refugee groups.

Testing Statistical Properties: Validity and Reliability There are many kinds of validity and reliability that must be demonstrated for a measure to be accurate across groups. For example, the HTQ reports the validity of the "greater than 2.5" average item score in predicting PTSD, but this was in outpatient psychiatric patients, and this score was not corroborated in a community based sample. Nevertheless, this cutoff score has been used in other community samples of refugees, assuming its validity. Data from broad community sources are important for testing validity because demand characteristics may bias data from specific populations, such as clinical samples or research volunteers.¹³⁴ For example, refugees who seek asylum may be motivated, even without awareness, to endorse high levels of trauma and symptoms to obtain refugee status or to please the interviewer.⁹⁴ Although the HTQ and VDS were developed to be culturally appropriate, they may have limited validity because they were developed and tested in outpatient psychiatry clinics. Validity and generalizability of a measure can only be estimated by including people who express the full range of events or health status; extrapolation of clinical data to refugees in general would be a "clinician's illusion."¹³⁵

There are few studies that have established good reliability and validity in some aspects of instruments in refugee research. For example, a cutoff score of 13 on the VDS demonstrated excellent validity to DSM-III-R major depression diagnosis in a community sample, and the HSCL-25 anxiety and depression scales demonstrate excellent test-retest reliability in Southeast Asian refugees. However, proper instrument development should demonstrate internal consistency (ie, item intercorrelation), stability (ie, consistent scores over time, such as test-retest reliability), and validity of the construct (ie, correlation with a standard). Instrument validity for psychiatric disorders is difficult to establish, since psychiatric diagnostic interviews as standards may be insufficiently valid.¹³⁶ Thus, determining what standards to use for validity testing can be a difficult methodological problem. Perhaps measures of impairment that are valid predictors of future health outcomes may be the best standards against which developed mental health measures should be tested.

There are limitations to this review. First, electronic searches of the literature are not error free, and citations to some studies and instruments may not be included in the literature. A recent meta-analysis of psychological consequences of forced displacement in Yugoslavia that searched 2 databases that we did not use (PsychLit and PILOTS) found 95 publications about refugee mental health, 12 of which were appropriate for their analyses and 10 of which our search did not initially find.¹³⁷ The only other review of refugee mental health that we located was published in 2000 and included only 12 studies.¹³⁸ In addition, our review was limited to English-literature publications. Second, it is difficult to extract accurate data from all publications. Some articles are difficult to obtain, some do not disclose all materials or methods used, and results are often unclear and difficult to interpret. Third, categorizing articles by type is partly subjective, and excluding articles about infectious diseases narrowed our review of instruments. Despite these limitations, our review indicates the need for improvements in the development, use, and reporting of instruments used to measure trauma and health status in refugee research.

Key Search Terms

Refugee:refugee, asylum seeker, internally displaced, externally displaced, war trauma, political violence, government oppression, ethnocide, governmental violence, ethnic cleansing, genocide, OR war population.

Trauma:war trauma, torture, human rights abuse, physical trauma, psychological trauma, OR psychological abuse.

Health status:health status, health outcomes, symptoms, syndromes, illness, disease, psychiatric disorder, psychiatric disease, psychiatric illness, mental illness, physical illness, physical disease, physical disorder, OR impairment.

Measurement:instrument, questionnaire, survey, measurement, assessment, OR psychometric.

Combination Operators

Refugee trauma:Refugee (all terms) AND Trauma (all terms)

Refugee health:Refugee (all terms) AND Health Status (all terms)

Final Combination

Refugee Trauma OR Refugee Health AND Measurement (all terms)

Funding/Support: This work was supported by National Institute of Mental Health grant R01 MH 59574.

Acknowledgment: We thank James Ruiz and Valorie Eckert, MPH, for helping to assemble material for this work.

(.)