Questionnaire-based mechanical exposure indices for large population studies - reliability, internal consistency and predictive validity

Questionnaire-based mechanical exposure indices for large population studies — reliability, internal consistency and predictive validity. Scand J Work Environ Health 2001;27(1):41—48. Objectives This study attempts to construct valid indices for mechanical exposure of the shoulder-neck region with relation to the development of shoulder-neck pain in a 1-year perspective study of a general population. Methods A comprehensive questionnaire was presented to 14 556 subjects aged 45 or 65 years and repeated after 12 months. Twenty-four questions concerning positions, movements, and manual materials handling were registered on a 3-point impact scale. Musculoskeletal problems were reported on a slightly modified version of the Standardized Nordic Questionnaire for the Analysis of Musculoskeletal Symptoms. Test-retest stability after 2 weeks was calculated for 232 consecutive participants. Based on mechanistic theories, 4 exposure indices were formed. Another 5 constructs were obtained by factor analysis. Results All the indices showed good test-retest stability, and 5 of them had very good internal consistency. Due to overlaps between the indices, 2 indices stood out as having unique properties. One of them concerned mainly postures and the other dealt primarily with measured lifting. However, the latter was not related to the shoulder-neck pain outcome when adjusted for the posture index. The posture index showed an exposure-effect relationship with the outcome. The job titles implied a large degree of exposure misclassification. analyses of interaction with other possible determinants of shoulder-neck pain (ie, psychosocial factors). The use of such an index instead of job titles in large population studies will reduce the risk of misclassification.

Although shoulder and neck complaints are common in many occupational settings, the quantitative relationships to environmental and individual factors are not fully understood (1). Associations with educational level, immigrant status, work organization, and mechanical exposure have been demonstrated (2)(3)(4)(5)(6).
Since shoulder-neck pain may originate from several local structures, a wide range of data is required for estimating mechanical exposure (6)(7)(8)(9). This is of particular importance for large populations with a mixture of exposures. When the relationship between complex exposure and the development of complaints is being studied, an index construct is essential for including this information in the analysis. A questionnaire comprised of many single items is the most feasible method with which to obtain exposure data in large populations (10).
Most previous index constructs have been based on observation methods, for example, the strain index, OCRA, and RULA (11)(12)(13). Observation methods, however, consume too many resources to be practical for use with larger populations. A validated questionnaire for estimating load has recently been published for the lumbar region, but there is still a need for methods to establish loads on shoulder and neck structures (14).
Procedures and guidelines for the construction of a mechanical exposure index have been poorly investigated. Only a few attempts have been described (6,15). Theoretical considerations regarding probable mechanisms for the development of complaints during exposure have guided the development of the index constructs explored by us in our present study.
The specific aim of our present analysis was to construct one or more valid indices based on self-reported mechanical exposure in the shoulder-neck region with relation to the development of shoulder-neck problems in a 1-year perspective study of a general population. The intended primary use of the index is for the analysis of possible interactions with psychosocial strain for the prediction of shoulder and neck trouble.

Design
The study was designed as a prospective questionnairebased open cohort study with a 1-year follow-up.
In 1990, all the subjects born between 1926 to 1945 and living in Malmö (a city in southern Sweden with about 240 000 inhabitants) were defined as the study base for the Malmö Diet and Cancer Study (16). The participants in this study between February 1992 and December 1994 were included in the Malmö Shoulder Neck Study (MSNS). The information given for recruitment to the study focused on the relation between diet and cancer, not musculoskeletal problems.
The study population was approached in the following ways: by postal invitation or by direct contact taken by persons after a mass media campaign. All the participants in the MSNS baseline study who were still registered in the municipality of Malmö were invited to participate in a follow-up 1 year later.

Population
The baseline study included 14 556 subjects, 6489 men and 8067 women. The participants did not differ significantly from the nonparticipants regarding sociodemographic data. Their mean age was slightly higher. There were relatively more women than men, and more of them were born in Sweden and were married (P-O Östergren personal communication). In this study group, 7365 (3674 women and 3691 men) were gainfully employed ≥30 hours/week. Most of them were born in Sweden, 254 were immigrants from other Nordic countries, and 504 were from non-Nordic countries.
The overall follow-up response rate was 87% (N=12 607). Of these subjects, 3624 (43% women) were at risk of developing a shoulder-neck disorder. They had had no shoulder-neck trouble during the last 12 months prior to the baseline study, were gainfully employed ≥30 hours/week, and had completed the follow-up questionnaire (response rate 89%).

Sampling procedures
The MSNS baseline questionnaire was presented to the participants during their 1st visit to the office of the Malmö Diet and Cancer Study. On their next visit, 2 weeks later, the questionnaire was checked for missing data, and, if necessary, complemented.
For the reliability analyses (test-retest), 232 consecutive participants were asked, during the spring of 1994, to fill out the baseline questionnaire once again after 2 weeks. Answers without missing values on the exposure items were obtained from 207 persons.
After a period of 12 months (median 12.6 months, interquartile 12.3-13.3), a 2nd questionnaire (see below) was sent to all the participants in the baseline study who were still registered in the municipality of Malmö. Information letters introduced the 2nd questionnaire, 2 reminders followed, and, finally, a telephone call was made, if needed. The questionnaires were immediately checked for missing data and, if necessary, complemented through a telephone interview.

Questionnaires
In the baseline questionnaire the current occupational mechanical exposure was estimated by 24 questions developed and used in the Stockholm MUSIC 1 Study, but the items on the time aspects of materials handling and walking distance were not included (17). Furthermore, the response scale was revised to a 3-point scale, as the ability to quantify the exposures in more detail has been found to be poor (17,18). The 3 levels of verbal quantifiers used in our study were hardly nothing or not at all, somewhat, and a great deal: score 1, 2, and 3, respectively (19). Thus the scale was related to the subjects' perception of the exposure quantity, and it did not consider the estimation of specific duration or frequency aspects. Table 1 presents the items on occupational exposure concerning body postures (11 items), repetitive movements (2 items), precision movements (1 item), vibration (2 items), manual materials handling (5 items), jumping and climbing (1 item), unexpected, sudden high loads (1 item), and level of physical activity (1 item). Complaints were assessed with a modified version of the Standardized Nordic Questionnaire, in which the original 5-point category scale, focusing on exact duration, was modified to focus on the perception of the temporal impact of the trouble (20). Questions were asked about problems (pain, ache, discomfort) in the neck, shoulders, combined elbows, wrist and hands, and the lower back. The response categories were no, never; yes, once or twice; yes, sometimes; yes, often; and yes, all the time (21).
The questionnaire also included instruments for assessing leisure-time activities, social and basic demographic variables, medical history, self-assessed global health and life-style factors, work demand and control, and social network. The questionnaire comprised 141 items.
The 2nd questionnaire repeated the questions for the assessment of exposures and the questions on shoulderneck problems. The latter were used to identify incident cases of shoulder-neck pain.

Criteria for index constructs
The following criteria were defined for the process of constructing and selecting the index to be recommended for interaction analyses. There should be good face validity (ie, intuitively reasonable content of both the single items and the index construct). The test-retest reliability of single items and the index should be at least good (ie, kappa >0.6) (21). The construct validity should be well argued. We preferred single items to be related to the incidence of shoulder-neck pain to avoid dilution of possible effects. This occurrence had, however, to be balanced against the risk of losing exposure information, which, in combination, would pose a risk of pain development. Furthermore, each index should mirror an exclusive construct, and single items should only be used in 1 index. The internal consistency, as measured by Cronbach's alpha, should not be less than 0.6 and preferably at least 0.8 (22).
The distribution of the index scores should reveal both a conceptually clear and a statistically feasible zero level of exposure. For calculating the exposure-effect relationships with good statistical power, the index scores should be dividable into at least 4 strata, each including a sufficient number of subjects. Finally, the index should reveal more information than when job titles are used as an exposure proxy.

Data analysis
Index scores were calculated as the sums of the single item scores. Data were analyzed with the statistical software SPSS® V8 for Windows® and Stata® V6.0. All the analyses of exposure data were made for subjects in gainful employment ≥30 hours/week. Cohen's kappa with quadratic weight was used as the measure of reliability (test-retest stability) of the single items and the indices. Index stability after 12 months was tested similarly. Cronbach's alpha was calculated as a measure of the internal consistency of the indices.
The same workplace factors have been shown as risk factors for both the neck and the shoulders (5). Table 1. Single items for mechanical exposure in the questionnaire and their initially proposed use in the index constructs -2-week test-retest stability (weighted kappa) and the relationships with 12-month incident shoulder-neck pain tested with odds ratios for each single item. The question posed was: "Does your work involve and require …?" (95% CI = 95% confidence interval, RFI = risk factor index, MI = mechanistic index, FI = factor analysis index) Therefore, in our analysis the subjects were classified according to the highest category of problems with either the shoulder or neck areas. For the exposure-response analysis, a 12-month incidence of shoulder-neck pain was defined by the change in the impact of the problem as "no, never" or "yes, once or twice" to "yes, often" or "yes, all the time". Thus the subjects listing the middle category "yes, sometimes" on any occasion were omitted.
Odds ratios for single items and strata of indices were used to test associations with the incidence of shoulder-neck pain. The odds ratios were conditioned for gender and other exposure indices in a logistic regression. The index quartiles of the total population defined 4 exposure strata. The exposure-effect relationship was tested by an analysis of trends with the use of a linear regression of index strata on the logarithmically transformed odds ratios. Because of a skewed distribution of the index score, all the comparisons between the subgroups were made on logarithmically transformed scores. A general linear model analysis of variance (ANOVA) was used for the analysis of 2-factorial models.

Index formation
Most of the single items had good or very good testretest stability (ie, kappa >0.6 or >0.8, respectively) (table 1). Only 3 items showed a kappa of <0.6. Most of the items were related to the incidence of shoulder-neck pain, and all of them were considered in the formation of possible indices (table 1). One risk factor index (RFI) was based on all the items. With the use of prior knowledge, the items were aggregated into 4 suggestions based on mechanistic considerations. During the data analysis 5 more factorial indices were constructed.
Mechanistic indices. Table 1 shows the 4 index constructs based on hypotheses concerning mechanisms of pathogenesis in work-related musculoskeletal disorders (23). A low force index (MI1) focused on long duration of constrained posture, which may cause, for example, trapezius myalgia. A medium force index (MI2) comprised mainly "poor" postures that may cause, for example, rotator cuff tendinitis. A high force index (MI3) was intended to measure extreme exposure amplitudes, for example, heavy lifting causing outcomes often classified as accidents. A high activity index (MI4) focused on high demands on central circulation.
Factorial indices. As a supplement to the indices suggested by a priori knowledge, a factor analysis of the 24 items suggested 5 possible constructs, called factorial indices 1 to 5 [FI1 to FI5 (table 1)]. The factor analysis was made with a principal component analysis with varimax rotation. Items with a factor correlation of ³0.5 were included in the constructs.

Job classification
From the 243 job titles in the study, the 20 representing more than 100 subjects each were used for comparison with the index scores. Based on experience and various published exposure data, 2 experienced raters independently grouped the job titles into 3 categories: low, intermediate, or high mechanical exposure. No discrepancy appeared between the raters. The distribution of the index score strata in each group was then used to describe the possible misclassification that would be a consequence of using the job titles only as exposure proxy.

Results
All 10 conceivable indices (ie, the risk factor index, the 4 mechanistic indices, and the 5 factorial indices) were submitted to a further test of their properties.
Choice of index. Of the initial 10 indices, MI1 and FI1 had poor internal consistency and were excluded from further consideration. There was a considerable overlap between MI2 and FI3 and between FI4 and FI5. The 3 factorial constructs had poorer test-retest and internal consistency properties than did MI2 and were therefore excluded from further consideration. A high degree of overlap existed between MI3, MI4, and FI2. The MI4-mirrored physical activity included all of the lifting items in MI3. Therefore we chose to include only the most complete of them (ie, MI4) for further consideration.
Very good test-retest reliability was achieved for the 3 remaining constructs (RFI, MI2, and MI4). RFI was hampered by the fact that the conceptual level of no exposure [ie, no item score reached level 3 (a great deal)] was only found for very few subjects. For MI2 and MI4, which were correlated (Spearman's rho=0.74), splits of RFI, the conceptually defined no-exposure level, 12 and 8, respectively, was equal to the lower quartiles of the observed index scores. Thus both the mechanistic indices, MI2 and MI4, fulfilled our preset criteria for an index of exposure to musculoskeletal load in the shoulder-neck region, while the RFI was more problematic due to its poor definition of no exposure.
The single item "lifting and carrying 1-5 kg" had been proposed for inclusion in both MI2 and MI4. As the MI2 reflected mostly postures and MI4 physical activity included lifting and carrying, we decided to use the lifting item in MI4 only.
The index selection process ended with a mechanical exposure index (MEI) (ie, MI2 without the item "lifting and carrying 1-5 kg", which was kept in MI4), conceptually a physical activity index (PHYI). The 2-week test-retest stability for the MEI was very good, as shown by a weighted kappa of 0.78. Cronbach's alpha was 0.83, which implied a very high internal consistency. The correlation between these indices was rather high (Spearman's rho=0.68).

Properties of the indices
Neither the MEI nor the PHYI scores were related to age. Men had slightly higher MEI scores (15.5 versus 15.2, ANOVA P=0.009) and clearly higher PHYI scores (10.2 versus 9.4, ANOVA P<0.001). The gender difference was of the same magnitude in all the age strata (ANOVA interaction P=0.5 and 0.4, respectively).  Table 3 shows that the scores were lower with increasing educational level (ANOVA P<0.001). Immigrants had higher exposure scores, and people of non-Nordic origin had the highest scores (ANOVA P<0.001). The significant interaction between educational level and ethnicity showed that immigrants were exposed to more musculoskeletal load than Swedes with similar education for both MEI (ANOVA interaction P=0.012) and PHYI (ANOVA interaction P<0.001).
The advantage of the MEI versus an ordinal exposure classification such as job titles is shown in table 4. Job title could, at best, discriminate between jobs with high or low mechanical exposure, while jobs in the intermediate level comprised a full range of reported exposure.
The predictive validity for the development of shoulder-neck pain during the 12-month follow-up is shown in table 5. There was a significant exposure-effect relationship between the exposure strata defined by the indices and the outcome, as indicated by the odds ratios. The effect of PHYI disappeared in the multivariate analysis regarding the relationship with incident shoulder-neck pain, as the adjusted odds ratios were reduced to equity. Finally, the stability of the MEI scores after 12 months was found to be very good (weighted kappa=0.83) for the 4112 subjects who stated that their work task had not changed during the follow-up period.

Discussion
Our study presents the construction and evaluation of an index of mechanical exposure of the shoulder-neck region in a large population with a diversity of occupations. The suggested index had "very good" test-retest stability and a conceptually clear "no exposure level". It was based on mechanistic considerations of the pathogenesis of pain in shoulder and neck structures. Thus, in our opinion, the index had good face, content, and construct validity a priori. Furthermore, it showed predictive capabilities, including an exposure-effect relationship, for the 1-year incidence of shoulder-neck pain. The index clearly demonstrated differences in the mechanical exposures related to occupational, educational, and ethnicity-based strata. It was not dependent on age and showed only slight gender differences, which, however, were unadjusted, for a possible influence of total pain load (24).
One crucial consideration in the construction of a questionnaire was the choice of the item-response scales. Earlier studies have shown that a 3-point scale had better test-retest stability than a 5-point scale for the self-assessment of work activities (17,25). We considered these experiences and chose to apply 3-step ratings on an impact scale to obtain optimal psychometric properties of the self-assessments. The quantifiers of the rating scales were formulated so that they implicated valid "no" and "significant" exposure, respectively (26)(27)(28).
The 24 single items had been chosen in agreement with the literature, and therefore good face validity was obtained for the MEI. Almost all of the items (21 of 24) had good or very good test-retest stability. Nineteen of the items had odds ratios with 95% confidence intervals above 1 for a relationship with the incidence of shoulder-neck pain, but all 24 were kept for possible inclusion in the index constructs due to the possibility of combined risk indication.
Job titles are commonly used as a proxy for exposure in epidemiologic studies. For the quantitative assessment of an exposure-outcome relationship, it is however necessary to estimate the risk predictors in more detail. Existing exposure classifications of occupations, based on observations combined with interviews, lead to an exposure profile, which does not yield the same resolution of exposure variability as our present suggestion (29)(30)(31). The Finnish job-exposure matrix was an ambitious attempt to make job title into a more precise estimate (31). In this method, 8 ergonomic factors were based on national work-environment surveys and modified by observational data and the experience of the assessor, ending with continuous scales on each supposed risk factor with the possibility to rank the studied occupations. The drawback is that these relatively precise estimates cannot be made on an individual level, but only in strata according to occupation. There is, however, a large interindividual variability within job categories, and therefore a risk of exposure misclassification of a substantial number of subjects is implied (32). Such a misclassification was obvious for job titles in our present study, as we found that 15% was exposed, according to our index score, in supposedly unexposed jobs, and, vice versa, 24% was unexposed in supposedly exposed jobs. A large variation has also been shown with electromyography and wrist angle measurements on subjects who performed exactly the same task (33). Thus only jobs with unambiguously low or high mechanical exposure can be distinguished by classifications based on job titles. This imprecision reduces the power of an analysis of the exposure-effect relationship, which may be improved by a more precise exposure index. Since mechanical exposure concerns several dimensions, several items are needed to capture it. There is a risk, however, of obtaining too wide a construct, which would lead to only a few subjects receiving scores compatible with "unexposed". That would have been the case, if we had used all 24 items. There was also a risk of indistinctiveness in the exposure-effect relationships when we used too many items. Item reduction was necessary to obtain consistent constructs with a well-defined no-exposure level.
We investigated 3 different strategies for index formation, the result being 10 potentially useful indices. Eight of them had the qualities needed for conceivable indices, but they were not independent of each other. Thus we found it possible to reduce the number of indices further to 2, which still showed some correlation.
The index suggested by Johansson (6)  Four factorial indices produced independently valid information, and therefore implied that there are several probable dimensions of exposure for the development of incident cases of shoulder-neck pain. We did not find it possible to use all these dimensions independently in a questionnaire study based on the general population, such as the Malmö Shoulder Neck Study. Since FI3, FI4, and FI5 each had unique properties and, taken together, corresponded to MI2, we found support for the choice of MI2. Furthermore, 11 of the 12 items included in MI2 were significantly related to the 1-year incidence of shoulder-neck pain. Thus almost all the a priori defined qualities of a useful mechanical exposure index were fulfilled. Similar properties were found for the physical activity index (PHYI). MEI reflects awkward work postures, static load, or high muscular strain during precise movements, while PHYI reflects manual materials handling including lifting. These 2 indices were rather highly correlated and, in a multivariate analysis, only MEI showed a significant relationship with the incidence of shoulder and neck pain. Consequently, posture was found to have greater importance for shoulder-neck incidence than did circulatory-demanding workload, such as lifting.
The usefulness of questionnaires for the assessment of data on physical workload has recently been emphasized (34). It was stressed that such instruments need further development to obtain more valid information. In our opinion, our study shows that our modifications of previously suggested items resulted in a high quality index for use in a prospective study of shoulder-neck pain.
We conclude that the mechanical exposure index derived from our study can be useful for studying the interaction between mechanical exposure of the shoulders and neck and other possible determinants for the development of shoulder-neck pain in populations with mixed exposure. The index was predictive regarding the 1-year incidence of shoulder-neck pain and had very good construct, content, and face validity.