Experts' subjective assessment of pesticide exposure in fruit growing.

OBJECTIVES
Exposure to pesticides in fruit growing was estimated by pesticide experts, occupational hygienists, and fruit growing experts to determine whether valid subjective assessments can be made by experts. The study objectives were (i) validation of exposure assessment by experts using different sources of information, (ii) assessment of interrater agreement, (iii) measurement of agreement between experts' assessments and actual quantitative exposure data.


METHODS
Three groups with different expertise made four ratings. Three of the ratings were made in three phases in which exposure information was provided.


RESULTS
The intraclass correlation was high for each subgroup of experts when tasks in fruit growing were relatively ranked by increasing exposure level. In general, the interrater agreement on factors influencing the internal dose decreased when more information on exposure was provided. Experts correctly considered dermal exposure as the prominent contributor to internal dose. Results were comparable for the three pesticides under study. The ranking of 15 specific sprayings with a fungicide clearly showed differences between raters according to their expertise. The pesticide experts and occupational hygienists were able to rank daily exposure levels during pesticide spraying in a meaningful way.


CONCLUSIONS
Experts seem to recognize the most important determinants of external exposure and therefore should be able pay a role in evaluating the effectiveness of control measures taken to reduce external exposure and to determine exposure groups in epidemiologic studies. The expert panel should not be too small, and consensus or average estimates should be used because differences within expert groups can be considerable.

Considering that a variety of factors can influence exposure and uptake into the body, exposure to pesticides in agriculture is complex. Over the last decade, increasing attention has been paid to the skin as a route of uptake. Data on skin exposure are difficult to interpret because of the lack of information on skin absorption under work conditions and therefore because of a lack of information on the contribution of skin absorption relative to inhalation and ingestion.
Epidemiologic studies often involve the effects of long-term exposures before the manifestation of disease. This type of exposure may result in problems because of the absence of adequate monitoring in the past (1). Job titles and job-exposure matrices, often developed for a specific industry, have been used to distinguish occupationally exposed workers in epidemiologic studies. Since occupational exposure to pesticides is complex because of the variety of compounds being used, a crop-exposure matrix was proposed (2). It relates agricultural practices to pesticide exposures and takes into account changes over time and the use of chemicals by farm area. Subjective assessments by experts to characterize occupational exposure have so far been used mainly in industry, and they have thus far focused on respiratory exposure (3)(4)(5)(6)(7). In some epidemiologic studies exposure assessments have been based on subjective assessments of exposure by individual study subjects (8-1 1).
To determine whether experts can estimate a complex situation of outdoor exposure in agriculture, pesticide exposure of fruit growers in The Netherlands was been estimated by 15 experts. As part of the concerted action "Retrospective Evaluation of Occupational Exposures in Cancer Epidemiology" in the European Community (12), cooperative studies were carried out in agriculture (vineyards), shoe and leather making and metal plating in Italy (13), and fruit growing in The Nether-Subjective assessment of exposure lands. This study has been carried out in close cooperation with the vineyard study of the Epidemiology Unit of the Department of Oncology in Torino, Italy.
The study objectives were (i) to validate exposure assessment by experts using different sources of information, (ii) to assess interrater agreement, and (iii) to measure agreement between experts' assessments and actual measured exposure data.

Study design and methods
Experts A panel of 15 experts was asked to assess exposure subjectively. The panel consisted of general occupational hygienists, occupational hygienists working at research institutes with experience in the field of pesticide exposure, and experts on fruit growing. Each group consisted of five raters. The three groups have been referred to in the report as occupational hygienists, pesticide experts, and fruit growing experts.

Information phases
Assessments were made after three consecutive information phases. During each of these phases information on fruit growing and pesticide exposure was presented. The assessments took place during a 1 -d meeting in November 1992. This approach was chosen because work in fruit growing concerns seasonal activities, most of which are weather dependent, so that inspection of the different work situations would not have been possible by visiting fruit farms. After each information phase, the experts were asked to make separate assessments for three different pesticides.

Ratings
Ratings were made in each of the three phases of the study. First external dermal and respiratory exposure was ranked according to performed tasks. The raters were asked to rank 14 tasks in fruit growing for external dermal and respiratory exposure according to performed tasks, attaching the ranking number 1 to the lowest exposure. The tasks were rated as follows: (i) mixing and loading liquid pesticide from 1-1 packages, (ii) mixing and loading powdery pesticide from 25-kg packages, (iii) spraying with a spray gun (10-1 tank), (iv) spraying with an airblast sprayer without a cabin; (v) spraying with an airblast sprayer with a cabin, (vi) tasks performed inside a shed, (vii) tasks with crops: pruning, (viii) tasks with crops: tying up, (ix) tasks with crops: bending, (x) tasks with crops: thinning, (xi) harvesting, (xii) transport of fruit from orchard to store, (xiii) sorting of fruit by hand, and (xiv) planting and treatment against fruit tree canker.
Then an estimation of the influence of different factors on the internal exposure dose was made. The 14 factors to be assessed concerned the influence of type of (spraying) equipment, personal protection equipment, and weather conditions. The factors potentially influencing exposure were cabin on tractor, high concentration in tank, large tank content, use of gloves, use of dust mask, use of respirator (dust or active coal filter), use of spraying suit, use of boots, head wind, side wind, wind behind, high wind speed, high temperature, and high humidity. The influence was categorized as follows: strongly reducing (--), reducing (-), no influence or irrelevant, increasing (+), or strongly increasing (++). The assessments had to be made for the application of pesticides and reentry (tasks performed in the orchards other than application) separately. The interrater agreement was expressed as Cohen's kappa. The kappa statistic has an interpretation as an intraclass corselation coefficient (14). For the case of multiple ratings per subject, the mean kappa was calculated as the weighted average, as proposed by Landis & Koch in 1977, cited by Fleiss (15). In case of equal numbers of ratings per factor, approximate standard errors for the kappa were calculated for testing the hypothesis that the underlying value is zero as proposed by Fleiss, Nee & Landis, cited by Fleiss (15). The intraclass correlation coefficient is a widely used measure of interrater reliability for the case of quantitative ratings. The five aforementioned categories were reduced to three by combining the "strongly reducing" with the "reducing" and the "increasing" with the "strongly increasing".
The relative contribution of the exposure route to the dose was also estimated. The raters assessed the relative contribution of exposure route to the total dose by giving a percentage for the dermal, respiratory, and oral contributions. The differences were studied using nonparametric statistics (Wilcoxon x2).
A random sample of 15 farms was taken from an exposure data base of 57 farms for the collection of data concerning the application of a fungicide (captan) with an airblast sprayer (table 1). A description was given of the main farm and task characteristics, without actual exposure data being presented. (See the appendix.) The characteristics of the individual captan sprayings on 15 de Cock et a1 fruit growing farms included crop type, mixing and loading, type of equipment, personal protection equipment, and weather conditions. The experts were asked to rank the farms for external dermal and respiratory exposure and the internal exposure dose, attaching a rank of 1 to the lowest exposure. An analysis of variance was used to calculate the intraclass correlation. The ranking was validated by calculating the correlation coefficients between the ideal (measured) ranking and the ranking made by the experts. The actual measured exposure was also compared between farms with a ranking of 8 or higher and rankings of 1-7 by a two-tailed t-test to determine whether the ranking resulted in groups of farms with different exposure levels.
Exposure was defined as contact during mixing or loading and spraying or during reentry or other work activities on the farm. External exposure was defined as potential exposure (ie, the amount of pesticide available for exposure) of the skin or respiratory tract without consideration of protection by factors such as personal protection equipment. The dose was defined as the amount of uptake by the body.
After a general introduction to the expert panel about the purposes of this study, and some general instructions, the experts were provided with information on exposure to pesticides in fruit growing in three information phases.
Phase 1 involved a 10-min video presentation describing fruit growing in general and factors which can influence pesticide exposure. In addition, methods for dermal (skin pads) and respiratory exposure assessment (personal air sampling) were shown. Next, slides were shown about activities such as the mixing and loading of pesticides, spraying, and the tasks performed in the orchards. Finally, written background information on occupational tasks in fruit growing and general information on three pesticides was given. In each phase assessments were made separately for three frequently used pesticides (captan, azinphos-methyl and hexythiazox). The purpose was to determine whether differences in the physical and chemical properties of the pesticides affected a rater's estimates. The pesticides represented fungicides, insecticides, and acaricides. Herbicides were not taken into account in this study, as they require different equipment for application and are used only infrequently. In phase 2 written information on dermal and respiratory exposure measurements together with the main farm characteristics and description of performed tasks was provided for six randomly sampled farms from the same data base used for rating involving the orchards treated with captan. Phase 3 used data for another six farms as described for the second phase. Actual exposure data were available only for captan.

Statistical analysis
Analyses were performed using Statistical Analyses System software. The rankings of the raters were correlated using the Spearman correlation. To study interrater agreement, Cohen's kappa (K) and the intraclass correlation coefficient (p), calculated by an analysis of variance (PROC NESTED), were used (16). PROC TTEST was used to study the differences in the mean exposure between the two exposure groups of ranked farms.

Ranking exposure according to performed tasks
The agreement between the experts (intraclass correlation) varied between 0.41 and 0.81. No differences in intraclass correlation were observed between the expert groups. Generally speaking the results were comparable for the different pesticides. For captan no differences between the information phases existed for the total group (N = 15), neither for dermal nor for respiratory exposure (table 2). However, the pesticide experts and occupational hygienists showed a small increase from Table 2. lntraclass correlation (p) between the raters in three groups of five experts in three information phases ranking 14 performed tasks in fruit growing for dermal and respiratory exposure for captan (method A). (p, = intra-class correlation as mean of k raters)

Influence of different factors on the internal exposure dose
For all three pesticides comparable results were derived. The initial interrater agreement after the first information phase was higher for the estimates based on spraying in comparison with the reentry activities (figure 1). Some differences were observed per expert group. For the fruit growing experts, the initial interrater agreement was highest for both the spraying (K = 0.75) and the reentry activities (K = 0.41) when this group was compared with the other experts. Information in general decreased the interrater agreement.

Relative contribution of exposure route to internal dose
The total group of 15 experts clearly changed their opinion after the first rating as far as dermal exposure was concerned. From the first rating phase, the highest rela- tive contribution to internal dose was attributed to the dermal route. After exposure data were presented, an even higher contribution was assessed. The pesticide experts gave dermal contribution the highest rating for all three tasks (mixing or loading, spraying, and reentry). Exposure data changed the initial assessment in favor of dermal exposure. The occupational hygienists initially assessed respiratory exposure as the main contributor to internal dose. The exposure data clearly changed their assessment on that point. Dermal exposure became more important, though less explicit, when assessed by the pesticide experts. The oral route was assessed as relatively more important by the occupational hygienists (contribution of around 20%) than by the other experts, especially during reentry of the orchard. The assessments of the fruit growing experts were in between the other expert groups. From the start this group also assessed dermal route as the most important. However, the exposure data did not influence their assessment significantly. Mutual comparison did not show distinct differences between pesticides.

Ranking of spraying activities
The intraclass correlation was higher for the pesticide experts and occupational hygienists than for the fruit growing experts. The highest intraclass correlation was found for the occupational hygienists (table 3). The intraclass correlation was higher for respiratory exposure than for dermal exposure. However, the validity for respiratory exposure was lower. Figures 2, 3, and 4 show the correlation between the ideal (measured) ranking and the ranking made by the experts for respiratory, total dermal exposure, and dermal exposure of the wrists, respectively. Generally, respiratory exposure was difficult to estimate. For nine raters' assessment of total dermal exposures, and for seven raters' evaluation of exposure of the wrists, the correlation between the rank number and actual measured exposure was statistically significant (P I 0.05). The  pesticide experts and occupational hygienists ranked spraying activities better according to actual exposure than the fruit growing experts did. The pesticide experts and occupational hygienists were also able to rank the farms into two groups with significantly different exposure. A statistically significant difference was found in actual measured exposure for these raters between the farms ranked as highly exposed and those ranked as lowly exposed (t-test, P 5 0.10, table 4).

Discussion
The information given to experts hardly influenced the intessater agreement in ranking tasks with respect to pesticide exposure. Perhaps the amount of information given to the experts in the second and third phase was not enough. In both phases they were provided with information on only six exposure situations. Another, more likely, explanation may have been that the information given in the second and third phase confirmed what the experts already knew. This possibility was supported by their ability to rank 15 spraying activities successfully according to dermal exposure.
The quantitative exposure information decreased the interrater agreement when the experts were asked to estimate the influence of 14 different factors on internal dose. In a study on semiquantitative estimation of exposure in a polyester factory, it was observed that the relative ranking of jobs was possible without knowledge of actual exposure levels (17). It was concluded that the chemical and physical properties can influence the ability to rank from low to high. A poor comparison of styrene with methylene chloride was explained by differences in physical properties such as the perception of smell. A study of Macaluso et a1 (18) concluded that expert-based estimates of solvent exposure appear to be feasible with the amount of information generally available for a specific plant. They considered the interrater agreement to be good (0.62-0.67) for cumulative exposure scores for frequent exposures. Initially, in our study only written information on physicochemical properties was provided to the experts. This information did not result in differences in the ratings. It should be noted that in the second and third phase only external exposure data for captan were provided without information on actual internal uptake.
In general, our pesticide experts and occupational hygienists changed their estimates of the relative contribution to the internal dose of different exposure routes when provided with data on external exposure. After exposure information was provided to the experts, they judged dermal exposure as more important. The occupational hygienists had initially judged the respiratory route as predominant, but changed their opinion after receiving more information. This occurrence may have been influenced by their education, which still mainly focuses on respiratory exposure. The fruit growing experts judged the dermal route as predominant from the start, probably because of their personal experience in fruit growing.
All the subgroups of experts were able to rank 15 dermal exposure situations meaningfully. The results for respiratory exposure showed smaller (nonsignificant) correlations between the ideal and the estimated ranking. The Spearman correlation coefficients for respiratory exposure were less than 0.50, and for dermal exposure for some experts it was around 0.90. Actual use of a cabin filter for four of the seven cabin users in this data set explained the lower respiratory exposure for these farmers very well. It was also one of the determinants of respiratory exposure observed in a large data set of sprayings (19). Obviously, the experts did not consider this feature of a cabin as a determinant of respiratory exposure.
Exposure data for reentry were not taken into account in the validation of the experts' estimates, since no measurement data were available at the time. For long-term exposure, however, reentry may contribute significantly to total exposure (19). Dislodgeable foliar residue has shown to be the most prominent determinant of exposure during reentry among fruit growers (19). As this information is not known when exposure is estimated retrospectively, one can doubt if expert panels are able to group workers retrospectively in a meaningful order. It is very unlikely that information on the concentration of the crop will be available retrospectively. In addition, information on reentry times should be known.
It was striking that dermal exposure estimation was possible for single exposure situations (point estimates). Nonetheless, the crucial question for dermal exposure is which skin areas are the most relevant for effects of long-term exposure and whether experts are able to recognize these skin areas. The experts were able to rank total dermal exposure and exposure of the wrists. Data from a study on the urinary excretion of tetrahydrophtalilnide (THPI) among fruit growers exposed to captan (20) showed the importance of dermal exposure. It was shown, however, that the highest exposed skin areas are not necessarily the most important areas for uptake. Therefore, the experts' judgment of dermal exposure may have been valid in itself, but it may be invalid for a retrospective assessment of uptake.
In addition, it would be of interest for retrospective exposure assessments, if experts were able to estimate past (dermal) exposure. It is likely that exposure during the application of pesticides declined, since application techniques changed and generally became better and tractor cabins were introduced to protect the fruit grower. It is not clear to what extent exposure during reentry may have changed over the years. The change may depend highly on changes in the application dose used, time spent on different tasks, and time elapsed since spraying before reentry.
The use of exposure estimates of an expert panel, estimating single-exposure situations, should therefore be combined with information on time spent on different tasks and information on the physicochemical properties of the relevant pesticides. The role of experts in the retrospective evaluation of occupational exposure to pesticides in epidemiology will need further validation. A study of Peyster et a1 (10) concluded that the combining of cholinesterase values with self-reported information strengthened the group classification of pesticide exposure in an epidemiologic study. Future attention should focus more on the relevant measure of exposure for a specific purpose (eg, the role of specific skin sites versus respiratory exposure); otherwise studies based on expert judgment may lead to highly spurious conclusions. As the population of fruit growers can be grouped according to determinants of potential external exposure (19) and experts seem to recognize the most important determinants, experts may play a role in evaluating the effectiveness of control measures taken to reduce external exposure and in exposure assessment in epidemiologic studies focusing on the health effects of exposure to pesticides. Preferably, the expert panel should not be too small and consensus or average data should be used because differences within expert groups can be considerable. The occupational hygienists and pesticide experts performed better than the fruit growing experts. There-