comparison of exposure estimates by worker raters and industrial hygienists.

A compari son of exposure estimates by worker raters and industrial hygienists. Scand J Environ Health 1989;15:424-429. The validity and reliability of nine senior sawmill workers' estimates of frequency, duration, and routes of exposure were compared with individual workers' ratings of their own job titles and industrial hygienists' ratings of all job titles in the plant. The reliability of the senior workers' mean exposure estimates [group intraclass correlation coefficient (ICC) = 0.71] compared favorably with those of three pairs of hygienists (group ICC =0.57, 0.1\7, and 0.81). The validity of their ratings was assessed in a comparison with urinary chlorophenate measurements representing 92 % of the job titles in the saw mill. The senior workers' ratings [coefficient of determination (R') = 0.22] resembled those of two pairs of industrial hygienists (R' = 0.24, 0.22)and was significantly greater than that of the third pair (R' = 0.08). The validity of the self-reports was also low (R,' = 0.15), but the difference was not statistically signifi cant. Senior workers' exposure ratings appear to be as effective as the other methods tested.

Documenting of historical exposures is a major difficulty encountered in the retrospective stud y of chronic occupational diseases. Quant itati ve dat a are rarely available from past hygienic surveys (I , 2). Generally, prior exposures must be estimated from histor ical data gathered at the time of the study. Estimates of past exposures are often made by indust rial hygienists or other occupational health professionals from within the study team (3)(4)(5)(6) or from the industry in question (3). Workers have also been asked to estimate their own exposures (4,7,8).
Our stud y was undertaken as part of a retrospe ctive cohort investigation of the mortality and cancer incidence of chlorophenate-exposcd sawmill workers. The only exposure-related.data about the sawmill employees which was routinely recorded and preserved during the study period was job title information. Methods used to translate job title data into exposur e estim ates in other studies were judged to be impracticable in the context of this study. No qua ntitative data about exposures in sawmill jobs were routinely collected until the end of the stud y period . Current data could not be used as a surrogate for past exposures because the exposure patterns changed during the period. Occupational health profe ssionals were not em-ployed in the industry until recently and were therefor e not familiar with earlier mill conditions. Finally, the study design did not involve contacting individual workers; therefore having each emplo yee estimate his own exposure was not feasible.
For these reasons, a new method of estimating exposure was developed. It used about 10 senior workers to rate the frequency, duration, and route of exposure for all job titles in the sawmill. Using comparisons to urinary chlorophenate levels and interrater agreement, we tested the validity and reliabilit y of the method in one sawmill (9). To explore further the method's operating characteristics, we extended the evaluation to another sawmill and report the results in this paper. Senior workers' estimates of curr ent exposure were compared with tho se of industrial hygienists and individual workers' self-reports. Althou gh neither of the latt er two groups could be used for retrospective exposure assessment in the cohort stud y, they represent theoreti cal alternatives against which our method should be jud ged.

Subjects and methods
The study sawmill emplo ys 268 hourl y workers in 73 job titles. Wate r-based solutions o f sodium tetra-and pentachlorophenate are applied in spray booth s in the planer mill and at the timberd eck in the sawmill itself. The mill has a large maintenan ce staff, many of whom may be exposed to the chemical. Other mill departments, including a boom area , dry kiln, powerhouse, loadin g shed , and lumber storage yard, have vario us potentials for exposure.

Measurement of chlorophenate expo sure
Chlorophenates, as used in sawmills, may be absorb ed both by inhalation and through the skin (10, I I), and they are largely excreted in th e urine (12,13). In order to establish a reference against which to evaluate the exposure estimates, we asked all the workers in the plant to volunteer urine samples over a seven-week period during the fall of 1987. Grab samples were collected on Thursda ys to reflect exposure over the previou s several weeks with some influence due to recent daily fluctuations, since estimates of the excretion halftimes of tetra-and pentachlorophenate vary from severa l da ys to about two months (14). All the samples were collected in silanized amber jars (15).
Th e ana lysis for total chloro phenates (tetra-and penta-) was carried out according to method 3412 of the Wor kers' Compe nsatio n Board of Brit ish Columbia (16). An aliqu ot of each sample was hydrolyzed, then neutralized and buffered. Tribromophenol was added as an internal standard. The solution was allowed to react with acetic anh ydride, and the acetylated phenol derivati ves were extr acted in isoo ctane. Th e halo genat ed phenols were quantitated with the use of a Va ria n 3300 gas chro matogra ph fitted with a Supelco SPB-5 capillary column and an electron capture detector.
Each urine sample volunteer was asked to compl ete a self-administered questionnaire about his usual job, actual jo bs held in the eight weeks before sampling, time of f in the eight weeks before the sampling, and the personal pro tective measur es normally emplo yed . Each worker was also asked to estimate his own exposur e over the previous eight weeks using the same questi on format as that an swered by the senior wor kers and hygienists estimating exposures for all job titles.

Estimation of chlorophenate exp osure by workers and hygienists
Fifteen senio r work ers, rand omly selected from tho se with more than five years ' seniority at the mill, were individually interviewed about the frequ ency, duration, and route of expo sure for every job in the sawmill. The intervi ewees were not informed of the co ntent o f the interview in adva nce, and the interviews were conduct ed in a controlled manner to pre vent interaction between the respondents. Of the 15 inter views, six were excluded on the basis of a prior i criteria, three because the workers were unable to answer questions about at least 85 (J"/ o of the job titles and three because the workers gave identical answers for all the job titles. The nine qual ifying senior worker rater s had held an average of I I jo bs in the mill and had an aver age of 19 years o f senio rity. Thre e pair s of hygienists were also interviewed. One pair inspected sawmills for a regulatory agency, two worked primarily in the lumber indu stry, including sawmill operations, and two were from other indu stries which do not include sawmill operat ions . Eac h 4 pair acted as a " rating team ," as has been used in other st udies (3,4,8). In preparation for the intervi ew, each team was given a list of the mill job titles and a list of the exposure rating que stions. Then each pair was given a tour of the sawmill during which they were allowed to ask qu estions o f any wo rker or supervisor. Th e tours lasted as long as each team felt necessary . Each hygienist was then interviewed individuall y in the same manner as the senior worker rat ers. Of the six industrial hygienists interviewed, one was unable to answer questions about 16070 of the jobs . In order to retain the pair stru cture, th at interview was retained for the anal yses of the hygienist groups.
Du ring the interviews, the job titles were pre sented in random order to pr event bias du e to the sequ ence of questionin g. The exposure questions about each job were obj ective in nature and had a co nsistent format. The frequen cy of exposure for a typical indi vidual in each job was categorized as daily, several times a week, several times a month, several times a year, or never. If frequency was rated as greater than never , the duration of expo sur e on a typical expo sure day was estimat ed as > 7 h, 5-7 h , 3-5 h, 1-3 h, and :5: I h .
Finall y, the route of expo sur e was rated as presence or ab sence of skin expo sure, vapor inhalation, and inhalation of contaminated sawdust. The frequency and dur ation estimates were multiplied to form a continu ou s exposure estimate in "hours of exposure per yea r" (9).

Data analysis
All dat a anal yses were performed using SPSS-X (SPSS = statistical package for the social sciences). The reliability of the raters' estimates of the chlorophenate exposure were assessed with the use of intraclass correlat ion coe fficients derived from a two-way anal ysis of varia nce (17,18). The validity of the rate rs' estimates was assessed with the proportion of variance explained (R' ) by a linear relation ship between the measured urinar y chloro phena te concent rations and the raters' mean estimates of expo sure for th e job title s.

Descriptive data
Urine samples and self-adm inistered que stionnaires were co mpleted by 226 wo rkers in 67 jobs, 84 0J0 of th e work for ce and 92 0J0 of the job titles being repr esented. One urine sample was excluded from the analysis becau se the donor was tak ing a chlorinated pharmaceutical for diabetes which co-chro matogrammed with tetrachlorophenate. The urinary chlorophenate concentratio ns of the remaining 225 samples ranged from 2 to 989 ug/l, with a mean of 99 ug/l, a geometric mean of 43 I1g/I, and a geometr ic standard deviation of 3.6. The distribution was approximat ely log normal; therefore the conc entrations were exp ressed as logarithms in all the an alyses.

Reliability analyses
The values for the intraclass correlation coefficient (ICC) of the rat ers' estima tes of chlorophenate exposure are reported in table I. Th e reliability analyses could not be performed for the self-reports since each urine sample volunteer rated only his own expo sure.
sample of 24 urine samples was reanal yzed by the laboratory where the analytic method was developed .
The results agreed with in ± 20 ug/I fo r 83 % of the samples , and no directional bias was found. Figure I illustrates the distributions of the job titl e expo sure ratings by the five gro ups of raters. All of the groups' expo sure estimates, except those of the hygienists from the lumber industry, covered the maximum range (0-2000 h/ year) ; those of the hygienists from the lumber industry ranged from 0 to 1000 h/year. The rater groups' mean estimates of exposure were less consistent, ie, 895 h/year for the senio r worker raters, 592 h/ year for the self-rep ort s of th e urine sample vo lunteers, 396 h/year for the government hygien ists , 371 h/ year for the other indu stry hygienists, and 69 h/ year for the lumb er indust ry hygienists. Both the volunteers making self-reports and the senior workers rated man y job titl es at the extreme va lues of less than 200 h/ year and great er than 1800 h/year. The hygienists rated most of the jobs at less than 200 h/year. The agreement between raters in a gro up, as assessed by the ' individ ual" ICC (17), was higher for all three ind ustrial hygienist pairs than for the senior wo rker raters, an expected result since the hygienists were in smaller groups and were able to consult with their team member.
Since it is th e gro up 's mean estimates which ha ve been used to assign exposures to the sawmill jo bs, the group ICC (18) is the ICC of mo st interest in the assessment of reliability. That is, if another group of similar raters had been selected, would their mean estimates of expo sure agree with tho se of the original group? As a result of their high level of individual agreement, the hygienists from the lumber industr y had the highe st gro up ICC as well. Howe ver, that of the senior worker raters exceeded those of th e oth er two hygienist gro ups. Thi s result may be explained by the larger number of raters in their gro up , a factor which would be expected to produce a more stable mean estimate than that of a smaller group with similar or even greater individual agreement levels.

Validity analys es
Ta ble 2 lists the proportion of variance (R') explained by a linear rela tion ship bet ween the volunteers' mea sured ur inar y chloro phenate concentrations and the rat ers' mean estimates of exposur e for the volunteers' job titles . For workers who held mo re than on e job, the rater s' exposure estimates were averaged for all job titles held in the eight weeks pr ior to the sampling (9). Two linear models of the relationship between the exposure estimates and the urin ary chlorophenate concentrations were constructed . One was based onl y on the estimates of hours of exposure per year, and the other also includ ed route of exposure ratings.
In the first model , the government hygien ists had the highest R2 , followed by the oth er industr y hygienists, and then the senior work er raters. The R2 of the urine sample volunteers and the lumber industry hygienists was con siderably lower, although only that of the latt er group was significantly lower (P < 0.05), as measur ed by a z-score com parison of the corre lation coefficients.
In mod el 2, the rate rs' answers to the question s on the rout e of exposure were included as separate ad ditive parameters. Answers " yes" or " no" to each rou te of exposure were assigned values of I and 0, respe ctively, an d then averaged for each rater grou p to give continuo us probability estimates for each rout e of exposure. In the case of the urine sample volunteers, who rated onl y their own exposure, the ro ute of exposure variables were dichotomous. Hours of expo sure per year was for ced into the mod el, and the rat ers' mean estimat es of skin, du st , and vapor expo sur e were entered stepwise only if they added a statistica lly significant increase in the explanator y power of the mod el (P < 0.05). The results were inconsistent. For two rater groups, no route of expo sure variables entered th e model. Skin was included in three cases, and vapor and du st were included once . The lumber industry hygienists , who had the poorest R' in th e simpler model, had the highest R' after all the route-of-exposure variables were added, a findin g suggesting that th is group relied mor e heavily on route-of-exposure information to distinguish between jobs. None of the model 2 R' values were significantly different from the oth ers.

Discussion
The interpretation of these results depends upon which comparisons are mad e. Although th e reliab ility results suggest th at the indu strial hygienists performed somewhat bett er fo r the individual ICC , the reliab ility of th e nine senior wor kers' rat ings as a gro up was comparable to those of the indu strial hygienist teams. The latter is the mo re appro pria te comparison since the gro up consensus values, not the ind ividual ratings, were used as the exposure estimate for each job. For the test o f validity, the senior worker rate rs fared as well as th e better hygienist team s.
How do these results compare to those found for the mill studied previously and to exposure estimatio n meth od s tested by ot her investigat ors? In th e earl ier study (9), the reliability of the senio r worker raters (individu aII CC = 0.51, gro up ICC = 0.91)] and the validity of th eir ratin gs (fall , all eight weeks, R = 0.72, R' =0.51) were higher. Several factor s may accoun t for this difference. The earlier mill had fewer jo b titles (N = 59), and th erefore it was mor e likely tha t each senior work er rat er was fam iliar with them . Th is possibility is pa rtly confir med by the fact th at on ly two interviews fro m this mill had to be excluded due to nonr espon se probl ems. Ano ther difference was the indu str ial relatio ns atmosphere at the mills. In the mill , N ote : Th e reponed ICC val ues differ somewhat f ro m those in t he earlier paper whe re a one-w ay, ins tead of t wo-way, analysis of variance was used to calcul ate th em .
used in our pre sent study, there was tensi on con cerning chlorophenate usc, a facto r which may have prompted three interviewees to give ident ical an swers for all jo bs. Th is situation did no t occur in the mill stu died earl ier. Goldberg et al (3) look ed at agreem ent in exposure coding by industry hygienists and by research-team coders. Average kappas for the agreement ranged from 0.51 to 0.69, with higher agreement within team coder pairs than within gro ups of six team coders or between team coders and indu stry hygienists. Kappa is a categorical an alogue to t he individual ICC. Those calculat ed for lumber and othe r industry hygienists in ou r study were in the same ran ge, as were th ose of th e senior worker raters at the mill studied earlier.
Several studies ha ve looked at th e relationship between raters' estimates a nd exposure measur ements. Woito witz et al (6) compar ed th e jo b expo sur e estimates made by members of the study team to dust measurements in the mining indu stry. doPico (7) compa red wor kers' estimates o f personal grain du st exposures to mea sur ed levels. In both cases, three hazard categories were used by th e raters, and the results were evaluated through comparison of the mean and standa rd deviation of the dust levels for jobs in each category. The mean du st levels had statistically significant increases with hazard class, but the standard deviations in each upper category overlapped the lower ones. Th is result cou ld signify either misclassificati on of certai n job titles, or it may reflect the greater variability in exposure in tho se jo bs with pot enti al for high expos ures.
A study by Kromhout et al (4) is th e most similar to ours. In five small facto ries, plant supervis ors a nd two indust rial hygien ists were asked to ra te all job titles using a four -point scale of expo sure . Workers in Table 2 each plant were asked to rate only their own exposure. Ratings were compared to the logarithm of the measured dust or solvent concentrations, and the results were reported as explained variability (R 2 ) in a oneway analysis of variance model. They concluded that the hygienists rated exposure better [mean R 2 = 0.28 (hygienist I) and 0.25 (hygienist 2)] than the others [mean R 2 = 0.16 (supervisors) and 0.22 (workers)]. The performance of the workers might be partly explained by the fact that they were rating only their own jobs, without considering its position relative to others in the plant. In both the mills we have tested, the selfreports of the urine sample volunteers had lower R 2 values than ratings of the senior workers, who considered all jobs.
An issue raised by Kromhout et al (4) is the effect of the proportion of exposure variability explained by differences between jobs, as opposed to within jobs. In one factory where the between-job variability accounted-for 77 1110 of the exposure variance, the raters performed better (R 2 = 0.37 to 0.58) than in other factories where between-job variability accounted for only 31 to 46 of the exposure variance (R' = 0 to 0.27). In the mill used in our present study, the between-job component of variance for urinary chlorophenate concentration was only 30 0J0, a value lower than that of any of the factories reported by Kromhout et al. This difference may explain the low R' values found in our study (for the nine senior workers 0.22 to 0.24). It is interesting to note that in the mill of our earlier study in which the proportion of variance explained was higher (0.51), the between-job variability was higher as well (42 0J0).
The question then arises of how the rest of the exposure variance might be explained. Much of the variance in our urine sample values occurred within a job and might have been related to an individual's use of personal protective equipment, their own location in the mill, days off, and use of chlorophenates at home. Since each urine sample volunteer was questioned about these factors, an additional analysis was conducted to determine their importance. Adding all personal protective measures (including hand washing on the job and use of protective boots, respirators, gloves, and aprons) to model 2 gave a statistically significant increase of 0.018 to the variance explained by the senior worker ratings. The small size of the increase is likely because there was little variation in these factors between workers with the same job. Self-reported exposure on the job and days off prior to sampling slight-1yincreased the explained variance, but the differences were not statistically significant. Use of chlorophenate at home added nothing, since so few workers reported such an exposure.
Other sources of variability are the exposure measurements themselves. As demonstrated by the reanalyses of our samples, there was a certain amount of variation in the analytical method for urinary chlorophenate. Many of the urine concentrations in the sam-428 pie from this mill were low, and therefore variability due to the method would have a profound effect on the ordering of job titles. In the mill of our earlier study, in which urine samples were collected during two seasons, the summer sample included more urinary chlorophenate concentrations greater than 100 ug/I, and the proportion of variance explained for this sample (all eight weeks, R = 0.76, R 2 = 0.58) was higher than for the fall. In addition, when the summer and fall samples were averaged, the proportion of variance explained by the exposure ratings increased by 0.104 over the fall sample by itself. The exposure raters were asked to make estimates based on a "typical" exposure, and, as such, the ratings should better correlate with urine concentrations averaged over a period of time than with any single measure of dose.
The results of this study indicate that senior workers' job exposure estimates should be equivalent in quality to those of industrial hygienists, and perhaps better than those provided by the self-reports of individual workers. Although more complex modeling of the exposure questionnaire data might allow a higher proportion of the variance to be explained, it may also be that job title information, which, as in many retrospective studies, is the only exposure-related data available for the sawmill worker cohort, must be viewed as an imperfect surrogate measure of dose. Some misclassification is inevitable, and therefore the inconclusive dose-related risk estimates must be interpreted with care.