Comparison of self-assessment and expert assessment of occupational exposure to chemicals

Comparison of self-assessment and expert assessment of occupational exposure to chemicals. Scand J Work Environ Objectives Occupational assessments of chemical exposure are often inadequate because of difficulties in obtaining sufficient numbers of measurements by trained professionals (experts). The objective of this study was to determine whether workers can provide unbiased data via self-assessments of exposure facilitated by the use of simple passive monitors for personal sampling. Untrained workers obtained personal measurements of their exposures to gaseous contaminants (terpenes in sawmills and styrene in reinforced plastics factories) with passive monitors and written instructions. To study the validity of the self-assessments, an occupational hygienist performed exposure measurements on the same occupational groups after the workers had obtained two or more measurements independently. The potential bias of the self-assessments was evaluated by comparing the self-assessments with the expert assessments in mixed-effects statistical models. Results A total of 153 terpene (97 self and 56 expert) and 216 styrene (159 self and 57 expert) measurements were obtained from four sawmills and six reinforced plastics factories, respectively. No significant differences in the geometric mean exposures were observed between the self-assessments and the expert assessments in 3 of 4 sawmills and 5 of 6 reinforced plastics factories (P > 0.10). The potential bias of the self-assessments of exposure ranged from less than 0.1% to 102% and was less than 17% in 9 of the 10 groups investigated. Conclusions The results indicate that untrained, unsupervised workers are able to collect consistently unbiased exposure data by employing currently available passive monitors.

A strategy for assessing chemical exposures should recognize the inherent statistical nature of the problem. It should also provide sufficient numbers of samples, including repeated measurements from the same workers, to estimate the within-and between-worker components of variance (1)(2)(3). Because traditional sampling methods rarely allow sufficient numbers of measurements to be collected, it has been suggested that the sampling design should include self-monitoring by workers (1,4). In four previous studies, involving occupational exposures to benzene (5), electric fields (6), welding fumes (7), and fuel additives (8), workers have collected personal measurements for assessing occupational exposures. We henceforth refer to such unsupervised sampling activities performed by workers as self-assessments of exposure (SAE).
We employed self-assessments to investigate exposures to terpenes in four sawmills and styrene in six reinforced plastics factories in Sweden. In each case, the workers obtained full-shift personal measurements with simple diffusive samplers, which had been distributed by mail. After the laboratory analysis of the monitors, Original articles the results were sent to both the workers and the supervisory staff for evaluation and interpretation. We had previously conducted a pilot study of the method to investigate benzene exposures among workers transporting gasoline (5). The aim of the current study was to compare the results of the self-assessments with those obtained by an occupational hygienist in the same workplaces to determine whether self-assessments can provide reasonable data with which to assess occupational exposures.

Workplaces and subjects
Four sawmills and six reinforced plastics factories in northern Sweden in the county of Västerbotten participated in the study. The sawmills were investigated between April 1997 and May 1998 and included no measurements during July 1997 (summer vacation). The sawmill workers were exposed to wood dust and gaseous components from the wood, including terpenes. Pine and spruce were the tree species primarily used in these sawmills. However, because pine releases many more terpenes than spruce, the sampling was only performed when the sawmills handled pine. The sawmills produced approximately 15 000-52 000 cubic meters of sawn board per year, of which 40-55% was pine. The tasks involved feeding the logs into the sawmill, sawing, edging, sorting, sweeping the floor, and sharpening saw blades. These activities generally took place in a single, two-story building, where workers rotated the tasks on a regular schedule. The number of participating workers varied between 7 and 11 per sawmill.
The workers in the reinforced plastics industry were exposed to styrene, which was a major component of the resin systems used to produce boats (3 factories), bathroom sinks (1 factory), security suitcases (1 factory) and a liner to repair walls of old pipes (1 factory). As with the sawmills, production was generally located in one building. These factories were investigated between December 1998 and February 2000 and included no measurements during July 1999 (summer vacation). The number of participating workers varied between 3 and 11 per factory. The tasks investigated in the reinforced plastics factories included lamination, gel coating, use of chopper guns, mixing of resins, casting of objects, and tool cleaning. The workers rotated these tasks on a daily or weekly schedule, depending on the size of the factory and the number of employees.

Air sampling and chemical analysis
Terpenes and styrene were collected as full-shift exposure measurements (minimum sampling duration of 360 minutes) by commercial diffusive samplers [stainless steel tubes, 90-mm × 6.3-mm outer diameter × 5.0-mm inner diameter (Perkin Elmer) containing approximately 300 mg of Tenax TA, 60-80 mesh (Chrompack)]. The samplers were conditioned at 300°C, sealed, and marked as described earlier (5). After the sampling, the monitors were returned to the laboratory by mail and were then analyzed within 4 days by thermal desorption and gas chromatography (GC). The tubes were desorbed as described previously, with a desorption temperature of 220°C for the monoterpenes and 250°C for styrene (5). The GC separation was carried out with a system similar to that used in an earlier study (5). The GC program for the terpenes was 120°C for 1 minute, then an increase by 10°C/minute to 200°C, and then by 20°C/ minute to 240°C, which was maintained for 1 minute. For styrene the initial temperature of 100°C was held for 1 minute, followed by a temperature rise of 10°C/ min to 200°C. The column pressure was 30 psi. A mixture of (±)-α-pinene, (1S)-(-)-β-pinene and (+)-∆ 3 -carene (ratio 10:1:5) dissolved in methanol was used to prepare reference standards for the quantification of terpenes, and styrene in methanol was used for the quantification of styrene, as described in an earlier study (5). All the chemicals used were > 98% pure.
The collection of styrene is well established for this type of passive monitor, using Tenax TA as the sorbent (9). However, because little was known about the collection of terpenes with Tenax TA, a validation study was done in parallel with this investigation. In that study, it was determined that the uptake rate of Tenax TA was lower at higher levels of terpenes (10). Thus the terpene levels were adjusted for the current study prior to the statistical evaluation, on the basis of data from an earlier study (10). The reported terpene values represent the sum of α-pinene, β-pinene, and ∆ 3 -carene.

Self-assessment of exposure
Oral and written information about the self-assessments was provided to both the workers and the management. It was simply indicated that the aim of the study was to determine whether workers could accurately measure their "full-shift" exposures. Detailed technical instructions indicated how to use the passive monitors. However, the instructions made no recommendations regarding measurements of "peak" exposure or to the selection of the days for monitoring; these decisions were left to the workers. A stock of samplers was always available in the sawmill, where the workers were encouraged to measure as often as they wished. At the reinforced plastics factories the workers were told to measure their Liljelind et al exposures at least four times during 4 to 5 months, and passive monitors were sent to them upon request.
During each day of measurement, the workers were requested to use three passive monitors simultaneously by placing them in a chest pocket oriented with the open ends extending out of the pocket. [Two monitors were typically analyzed, the third monitor only being analyzed when a mishap occurred with one of the first two. The estimated mean from the two measurements was used for the statistical analysis.] On tags attached to the samplers the workers occasionally provided comments about the activities carried out on the days of monitoring. These comments were not included in the current statistical analyses. The monitors were opened at the start of the workshift and closed at the end of the shift. Although workers were not requested to close the monitors during the lunch break, they were asked to note if the break lasted for 30 minutes or 1 hour. [Air concentrations were subsequently adjusted by subtracting the duration of the lunch periods.] Because workers in the reinforced plastics factories occasionally wore respirators (during spraying of the resins, usually in a ventilated booth), they were requested to close the passive monitors during these periods and to note when this type of situation occurred.
Within a week of the analysis, the results of the personal monitoring were sent to the workers, in the form of a diagram showing all his or her measurements to date, along with information about the current occupational exposure limit (OEL). Along with the results, a booklet was included with information about the variation in exposure levels to be expected from day to day and the telephone number of the participating occupational hygienist who could answer any questions. The booklet did not include any recommendations for preventive measures, for further sampling, or for reporting values to supervisory personnel. Summaries of all the measurements at a given facility were periodically sent to management in the form of diagrams showing individual measurements identified by the workers, together with the OEL; it was also indicated that a variation in air concentrations is expected from day to day in workplaces.
Two months after the introduction of the self-assessments of exposure and at the conclusion of the study of the sawmills and periodically during the study of the reinforced plastics factories, both management and the workers were interviewed by a psychologist about the interpretation of the measurements and the persons' perceptions of the self-assessments as a means of evaluating occupational exposures (reported elsewhere).

Expert measurements
Expert measurements were obtained on randomly selected days by a trained occupational hygienist. [None of these measurements were obtained the same day that the workers performed the self-assessments.] With minor exceptions, the occupational hygienist did not conduct personal monitoring of a given worker until at least two self-measurements had been made. Expert measurements employed the same diffusive samplers and procedures as had been recommended for the self-assessments, except that the occupational hygienist was present at the workplace and made notes of observations during the workshift. The expert opened the passive monitors, affixed them to each worker's lapel, capped the monitors after the sampling, recorded the time, and transported the passive monitors to the laboratory.

Statistical analysis
All the statistical procedures were performed with SAS (statistical analysis system) software PC 6.12 (SAS Institute, Cary, NC, USA). Because of obvious skewness and heteroscedasticity, the analyses were performed using the natural logarithms of the air concentrations. Nested mixed-effects models (Proc Mixed) were used to investigate the fixed effect of self-assessment of exposure (SAE) versus expert assessment of exposure and to estimate the variance components associated with the random effects of the worker (between-worker variability) and the error term (within-worker variability) according to Rappaport et al (7). Mixed-effects models provide a convenient structure for evaluating the magnitudes of fixed effects (group mean exposures and effect of self-assessments of exposure) while accommodating for the correlated errors inherent in the repeated measures design, where multiple measurements are obtained from the same persons. The effect of self-assessments of exposure is designated here as a fixed effect because we wished to make inferences about geometric mean exposures, which were measured either by the workers or by an expert in a specific set of worksites.
Analyses were performed separately for the terpene exposures in the sawmills and the styrene exposures in the reinforced plastics factories (hereafter, simply "factories") using the following model: for h = 1, 2, … g groups (sawmill or factory), for SAE = 0 for expert assessment of exposure and 1 for self-assessment of exposure for i = 1, 2 … k h workers in the h-th group, and for j = 1, 2 ….n h(i) measurement of the i-th worker in the h-th group.
In this model X h(ij) represents the exposure level on the j-th day for the i-th worker in the h-th group (with the parentheses emphasizing that workers and days are nested within groups); Y h(ij) is the natural logarithm of X h(ij) . Under this model, Y h(ij) is determined by the sum measurements or expert measurements, the mixed model was applied separately to a subset of workers having both self-measurements and expert measurements (185 of 216 observations in the factories and 143 of 153 in the sawmills).
The fits of the final models were evaluated by investigating the normality of the standardized predicted random effects (obtained from Proc Mixed output) using the Shapiro-Wilks test at a significance level of 0.05, as well as visual inspection of a q-q plot representing expected and observed quantiles (via Proc Capability). In each case, the random effects (based upon an analysis of logged exposure data) were judged to be adequately described by normal distributions.

Summary of the air measurements
The data available for the statistical analysis are summarized in table 1. Altogether 153 terpene and 216 styrene exposure measurements, collected from 36 sawmill workers and 42 factory workers, respectively, were deemed suitable for analysis. A few additional measurements (7 for terpenes and 5 for styrene) were excluded from the statistical analysis because of an inappropriate sampling duration (less than 360 minutes) or a lack of information about duration recorded on the sampler tag. In addition, three styrene-exposed workers were excluded (23 measurements) because they worked in rooms or buildings that differed from the remaining persons at a given factory. According to interviews with all the workers in the sawmills, 8% of the workers declined to participate. In the reinforced plastics factories the corresponding rate could not be estimated because all the workers were not interviewed. The summary of the minimummaximum numbers of measurements shown in table 1 indicates that there was no apparent difference in sample frequency between the sawmills and plastics factories.
Since most of the data (63.4% for terpenes and 73.6% for styrene) were collected by self-assessments, it was possible to obtain relatively many repeated measurements (terpenes: 1-13 per worker, median = 4; styrene: 1-9 per worker, median = 6). Only one or two expert measurements were generally obtained from each worker and, in a few cases, no expert measurements were available for practical reasons (eg, workers had left employment or were sick on the days of the expert assessment). Likewise, in a few cases, only expert measurements were available.
Scatterplots of the data are presented in figure 1. The ranges of the exposures were large (ie, between 3 and 813 mg/m 3 for terpenes and between 0.4 and 245 mg/m 3 for styrene). The scatterplots also indicate some differences of µ yh , representing the fixed underlying mean exposure (log space) for the h-th group (under expert assessment), α SAE , representing the fixed effect of the self-assessments of exposure, β h(i) representing the random effect of the i-th worker in the h-th group, and ε h(ij) representing the effect of the j-th day of the i-th worker in the h-th group. It is assumed that β h(i) and ε h(ij) are normally distributed and independent with means of zero and the variances of σ 2 B,h and σ 2 W,h of the logged exposure, representing the between-and within-worker variance components, respectively. The purpose of applying the mixed model was to determine whether the fixed effect of the selfassessments (α SAE ) was significant in each sawmill or factory, indicating a difference in the group mean of the logged exposures between the self-assessments and the expert assessments. In natural space a significant effect of the self-assessments would indicate that the geometric mean exposure of group h (ie, e µ y,h ) differed between the worker (self) and expert collection.
In fitting the model (separately) to sawmill or factory data, likelihood ratio tests were performed at a significance level of 0.05 to determine whether it was appropriate to pool σ 2 W,h among the groups, consistent with the model recommended by Rappaport et al for a similar study (7). The results indicated that σ 2 W,h could be pooled among sawmills 2-4 and among factories 1, 3 and 5. Thus the effect of self-assessment (α SAE ) was tested on the assumption of distinct σ 2 B,h and common σ 2  The estimated means and ranges of the measurements are summarized in table 2 for the self-assessments and expert assessments at each sawmill or factory. With minor exceptions (eg, sawmill 1 and factory 4) the estimated mean values were similar for each group regardless of whether self or expert data were used.

Mixed models
The REML estimates are summarized in table 3 for the parameters that defined exposure for each group. Note that the estimated within-worker variance component, σ 2 W,h , was common for three groups of sawmill workers (sawmills 2-4) and three groups of factory workers (factories 1, 3 & 5) and was distinct in all other cases. The estimates of σ 2 B,h and σ 2 W,h covered wide ranges with σ 2 W,h exceeding σ 2 B,h in most cases, consistent with results from other occupational settings (3,11). The estimated effects of self-assessment α SAE , shown in table 3, were not significant (P > 0.05) for any of the groups except for factory 4 (P = 0.02); in addition, this effect was of borderline significance for sawmill 1 (P = 0.06). In a comparison of the estimated parameters in table 3 from all the workers with those from only workers with both self-measurements and expert measurements, no major differences were observed.
If expert assessment is assumed to be unbiased, then any potential bias of self-assessments can be evaluated by dividing the estimated effect of self-assessment, α SAE , by the group mean, µ y,h , which had been estimated under expert assessment. The last column in table 3 compiles these estimates of self-assessment bias, which for all the groups ranged from less than 0.1% to 102%, 9 of 10 values being less than 17%. For only workers having both self-measurements and expert measurements, the estimated bias ranged from less than 0.1% to 54.6%, 9 of 10 values being less than 8.1% Table 2. Estimated means and ranges of the measurements obtained by self-assessment and expert assessment in each group.

Discussion
As noted in the introduction, very few occupational studies have involved workers in the assessment of exposure (5)(6)(7)(8). Of these, only our pilot investigation of benzene-exposed truck drivers (5) and the study of electric-field exposures in the electric-power industry (6) employed simple, passive personal monitors and fullshift sampling (the other studies used active samplers over portions of the workday). In the study of electric fields, only 3% of the workers refused to participate and 8% of the measurements were invalid due to lost meters and procedural errors (6). Likewise, in our study the participation rate was high in the sawmills, where 8% of the workers declined to participate and the proportions of invalid measurements were 6% for terpenes and 2% for styrene. [The participation rate in the reinforced plastics factories could not be estimated because not all the workers in these factories were interviewed.] On the basis of our results and those of Loomis et al (6), we suspect that the ease of measurement in these investigations increased the participation rate of the workers and reduced the numbers of invalid measurements. To our knowledge, no prior studies have compared self-assessment with expert assessment with the aim of determining possible bias associated with self-assessments of exposure. In our investigation, measurements were conducted by an occupational hygienist in parallel with those obtained by the workers independently. On the basis of the mean (logged) exposure for each group (µ y,h ) (or the corresponding geometric mean), our results indicate that the effect of the self-assessments was not statistically significant in 9 of the 10 groups investigated (P > 0.05) and, with the exception of the styrene exposures in factory 4, that any potential bias of the self-assessments was less than 17% (table 3). In the case of factory 4, where the self-assessment bias was estimated to be 102%, much of the discrepancy between self-measurements and expert measurements involved a single worker who was no longer employed when the expert measurements were collected (see figure 1). When only workers having both self-measurements and expert measurements were included, the estimated bias of factory 4 was 54.6%. We conclude that, even though the workers were not advised about when or how often to sample (no recommendations were made in the sawmills and workers were merely instructed to make at least four measurements in the plastics factories), the measurements obtained by self-assessments were almost entirely consistent with those obtained by the occupational hygienist on randomly selected days.
Among a set of general needs for health surveillance, recently given by Kauppinen & Toikkanen (12), was a recommendation for the development of cost-effective measurements and survey strategies. Our results indicate that self-assessments can be instituted whenever simple passive monitors are available for personal monitoring, as is generally the case for gaseous air contaminants. With recent advances in the development of passive monitors for aerosols (13)(14)(15), we expect that it will soon be possible to extend self-assessments to hazardous dusts as well. If self-assessments were applied to working populations over several years, it would be possible to achieve sufficient sample sizes to eliminate many of the problems currently plaguing epidemiologic studies, which invariably suffer from a lack of exposure data. Certainly, the statistical tools (mixed models) needed for evaluating such data are readily available, as we (and others) have shown. Finally, we think that self-assessments of exposure serve another important function; namely, they involve workers directly in the process of exposure assessment and control. With appropriate mechanisms for obtaining additional information about tasks and practices during the days of measurement, it will be possible to determine the effects of such covariates upon exposure. Obviously, such information will be useful for educating workers, as well as management, about potential problems and will assist the development of appropriate controls. In order to facilitate the effective î î î î î î î î î î Table 3. Parameters of exposure to terpenes and styrene estimated on the basis of logged data. [1 = estimates based on all measurements, 2 = estimates based on only measurements from workers who had both self-measurements and expert measurements, µ y,h = estimated mean exposure (log space) for the h-th group under expert assessment, α SAE = estimated fixed effect of the self-assessment of exposure, σ 2 B,h = estimated between-worker variance component of the logged exposure in the h-th group, σ 2 W,h = estimated withinworker variance component of the logged exposure in the h-th group]

Agent
Mean exposure group Between-person variation Within-person variation Self-assessment effect Self-assessment bias (  The mixed model assumed a distinct value for, σ 2 B,h and a common value for, σ 2 W,h . * 0.05 ≥ P < 0.10. ** 0.01 ≥ P < 0.05.
Liljelind et al involvement of workers in the process, it will be important to ensure timely feedback of results so that they can relate measurements to recent experience. Again, our study suggests that this is entirely possible with current technology.