The Danish Psychosocial Work Environment Questionnaire (DPQ): Development, content, reliability and validity

Psychosocial Work Environment Questionnaire (DPQ): Development, reliability and Objectives The aim of this study was to describe the development and the content of the Danish Psychosocial Work Environment Questionnaire (DPQ) and to test its reliability and validity. Methods We describe the identification of dimensions, the development of items, and the qualitative and quan titative tests of the reliability and validity of the DPQ. Reliability and validity of a 150 item version of the DPQ was evaluated in a stratified sample of 8958 employees in 14 job groups of which 4340 responded. Reliability was investigated using internal consistency and test-retest reliability. The factorial validity was investigated using confirmatory factor analysis (CFA). For each multi-item scale, we undertook CFA within each job group and multi-group CFA to investigate factorial invariance across job groups. Finally, using multi-group multi-factor CFA, we investigated whether scales were empirically distinct. Results Internal consistency reliabilities and test-retest reliabilities were satisfactory. Factorial validity of the multi-item scales was satisfactory within each of the 14 job groups. Factorial invariance was demonstrated for 10 of the 28 multi-item scales. The hypothesis that the scales of the DPQ were empirically distinct was supported. The final DPQ version consisted of 119 items covering 38 different psychosocial work environment dimensions. Conclusions Overall, the DPQ is a reliable and valid instrument for assessing psychosocial working conditions in a variety of job groups. The results indicate, however, that questions about psychosocial working conditions may be understood differently across job groups, which may have implications for the comparability of questionnaire-based measures of psychosocial working conditions across job groups.

Self-administered questionnaires are the most widely used method to measure psychosocial working condi-tions. Some questionnaires measure selected aspects of the psychosocial work environment based on a distinct theory hypothesizing that these aspects are important for specific outcomes, such as the health and well-being of employees or labor market participation. Well-known examples are instruments to measure job strain (28), effort-reward imbalance (29), organizational justice (30), workplace social capital (31,32), or illegitimate job tasks (33). Other questionnaires are not limited to a distinct theory but offer comprehensive measurements of the psychosocial work environment. Examples are the National Institute for Occupational Safety and Health's Reliability and validity of the DPQ (NIOSH's) Generic Job Stress Questionnaire (34), the General Nordic Questionnaire for Psychological and Social Factors at Work (QPS-Nordic) (35), and the Copenhagen Psychosocial Questionnaire (COPSOQ) (36,37). In particular, the COPSOQ has become a widely used instrument for comprehensive assessments of the psychosocial work environment, both in Denmark and internationally (38)(39)(40)(41)(42)(43).
In this article, we present the development and validation of a new questionnaire for the comprehensive assessment of psychosocial working conditions -the Danish Psychosocial Work Environment Questionnaire (DPQ). The DPQ follows the same basic principles and theoretical considerations as the COPSOQ, namely that the questionnaire should (i) be theory-based but not based on one single theory, (ii) inquire into psychosocial working conditions that are located at different organizational levels in the workplace (eg, individual, group, and organizational), (iii) be comprehensive by focusing on a variety of factors in the psychosocial work environment, and (iv) be directly applicable to all types of jobs (36).
A central focus of the COPSOQ was the continuous development of the questionnaire to keep its thematic profile up to date with developments in the psychosocial work environment (36). When we started our work, the latest COPSOQ revision (COPSOQ-II) dated back to 2005, and we aimed to update the COPSOQ-II questionnaire. During this work, however, we departed from COPSOQ-II to such an extent that we decided to rename the questionnaire. After we had completed the DPQ, the international COPSOQ-network published an updated COPSOQ-III questionnaire that is available on the network's homepage (www.copsoq-network.org). We will address the similarities and differences between the DPQ, COPSOQ-II and COPSOQ-III in the Discussion section of this article.
This study describes the development of the DPQ and aims to examine the reliability and the validity of the measures in the questionnaire. The full English version of the DPQ can be found in e-Appendix 1 (www.sjweh. fi/show_abstract.php?abstract_id=3793).  Table 1 gives an overview over the four phases of the development process: (i) identification of dimensions

Phase 1: Identification of dimensions and development of items
We scrutinized the COPSOQ-II questionnaire, data from the COPSOQ-II validation study (37), and the literature on its psychometric properties (37,(45)(46)(47) to identify dimensions and single items to be retained in the DPQ. We scanned research articles published in 2010-2013 in 20 peer-reviewed journals (e-Appendix 3, www.sjweh.fi/show_abstract.php?abstract_id=3793) and 50 psychosocial work environment questionnaires (e-Appendix 4, www.sjweh.fi/show_abstract. php?abstract_id=3793). Moreover, we conducted semi-structured interviews with 53 employees in 16 workplaces covering the five main occupational sectors in Denmark (e-Appendix 5, www.sjweh.fi/show_abstract.php?abstract_id=3793) to (i) assess the relevance of the dimensions in COP-SOQ-II, (ii) identify emerging issues and (iii) understand how employees in different sectors talked about their psychosocial work environment. Finally, we discussed a draft questionnaire and the findings from the interviews conducted in phase 1 at two meetings with an international advisory group.
On the basis of the activities in phase 1, we identified 34 dimensions of the psychosocial work environment to be assessed from multi-item scales or single-items. The 34 dimensions were operationalized by 130 items. We grouped the 34 dimensions according to their content in five overall domains of the psychosocial work environment using a modified version of the categorization system previously applied in the COPSOQ-II (37): (i) demands at work, (ii) work organization and job content, (iii) interpersonal relations: cooperation and leadership, (iv) conflicts in the workplace, and (v) reactions to the work situation. Items were grouped in scales and dimensions were grouped in domains on the basis of conceptual and theoretical considerations.
Phase 2: Qualitative test of the DPQ Using the draft questionnaire developed in phase 1, we conducted qualitative interviews with 26 employees in 13 of the 16 workplaces that participated in the focus group interviews performed during phase 1. The aim of the interviews was to assess (i) whether the items of the questionnaire were intelligible, (ii) whether the informants understood the questions as intended, and (iii) whether the content of the questionnaire was deemed relevant by the informants. In the interview, informants were shown cards with the items constituting each dimension of the psychosocial work environment. After reading each card, informants were asked to comment on the relevance and the intelligibility of the dimension and the items. Based on the results of these interviews, we revised several items in the questionnaire. The revised questionnaire was then presented to Danish work environment researchers, experts from the Danish Working Environment Authority and representatives from Danish unions and employer associations.
Phase 2 resulted in a test questionnaire with 150 items operationalizing 38 dimensions of the psychosocial work environment. Next, using the test version of the DPQ we tested the reliability and validity of the DPQ in phase 3.

Phase 3: Quantitative test of the DPQ
Study design and population.We tested the DPQ in a stratified sample of 8958 individuals employed in 14 different job groups. These job groups were selected to obtain stratification by educational attainment (low, medium, high) and primary work-task (knowledge work, client-related work, work related to production and transportation, and sales work) of the respondents. Specific job groups were selected to represent each stratum. In the category "low education, work related to production and transportation", we selected two different job groups (mail carriers and slaughterhouse workers) due to the highly differentiated types of jobs in this category. Further, we added the job group "police officers" to the sample as this job group has the direct possibility to apply force towards the clients they deal with, distinguishing this job group from the three other job groups representing client-related work tasks. Accordingly, the stratified study population represents a range of different occupational positions in the Danish labor market in terms of educational attainment and primary job tasks (table 2). Within each job group, employees were randomly drawn from a national register on income and labor market attachment for all persons in Denmark (e-Indkomstregistret).
Based on previous research, we aimed to obtain approximately 300 responses in each of the 14 job groups. As we knew from earlier surveys that response rates tend to vary across job groups, we investigated the response rates for the 14 selected job groups in a previous Danish work environment survey (48). From these expected response rates, we calculated how many potential respondents we should invite to obtain the required 300 responses in each job group (table 3).
The 8958 employees were sent an invitation letter by postal mail with instructions on how to access the online questionnaire. Non-responders were contacted by another letter and up to two times by telephone to obtain response. A final reminder was sent by postal mail together with a paper-and-pencil questionnaire. The data collection took place from April to June 2015. We obtained responses from 4340 individuals, yielding a 48.4% response rate. The response rate varied between 35.3-61.6 across the 14 job groups (table 3). A non-response analysis in the 14 job groups showed that women were significantly more likely than men to respond in six job groups (private bankers, sales assistants in shops, primary school teachers, healthcare helpers, teaching and research in universities, and office workers) whereas we found no significant sex-differences in the response pattern for the remaining eight job groups. In all 14 job groups, older individuals were significantly more likely to respond than younger individuals (e-Appendix 6, www.sjweh.fi/show_abstract. php?abstract_id=3793).
To conduct test-retest reliability analyses, we asked respondents to disclose their e-mail address. Three weeks after their response to the baseline questionnaire, the 1589 respondents who disclosed their e-mail addresses received a follow-up questionnaire. Of these, 660 responded (response rate: 41.9%, table 3).
Measures. The test version of the DPQ used in the quantitative test contained 150 items that operationalized 38 different dimensions (through multi-item scales and single items) of the psychosocial work environment that were grouped in five overall domains.
Of the 150 items, 27 were identical with items from COPSOQ-II and 19 were modified items from COPSOQ-II, 8 were identical with items from a Danish labor market survey "Work environment and health in Denmark" (48) and 2 were modified items from this survey, 4 were adapted from the QPS-Nordic (35), and 1 item each was respectively adapted from a question-naire by Sasser & Sørensen (49), the Work Design Questionnaire (50), and the Danish National Working Environment Survey (DANES) (51). All 9 items from the scale work engagement were translated from the Utrecht Work Engagement Scale (UWES) (52, 53) and 9 items were adapted from a Danish questionnaire on workplace social capital (32). Dimensions and sample items are presented in table 4. For a complete list of all items and more detailed references see e-Appendix 1, www.sjweh.fi/show_abstract.php?abstract_id=3793.
In addition to the psychosocial work environment items, the test questionnaire included 1 item assessing the physical environment, 17 items on the respondent's background information and 31 items measuring potential outcomes, such as self-rated health, depressive symptoms, and work ability. Age, sex and job group of participants were retrieved from national registers.
With the exception of the 6 items within the domain conflicts in the workplace, all items used ordinal categorical response options. Scale scores were calculated by recoding item scores from 0-100 and averaging the scores for items within each scale. For each scale, the score of 100 indicates the highest level of the measured dimension (e-Appendix 1).
Statistical analysis. The internal consistency reliability of the scales was assessed using the Cronbach's α coefficient. This analysis was based on the 4340 respondents who participated in the baseline survey. We calculated α-values for all scales with ≥3 items and deemed values >0.70 a satisfactory level of internal consistency. We calculated α-values for the entire study population and for each job group.
We assessed the test-retest reliability using partial intra-class correlations (ICC) (54), adjusted for the job group of the participants. The analysis of test-retest reliability was based on the 660 respondents who participated in the baseline and retest surveys. We deemed a partial ICC >0.70 a satisfactory level of test-retest reliability (46).
We deployed different modes of analysis to investigate the construct validity of the measures. First, we calculated means and standard deviations of the scales for each job group. Second, we assessed the factorial validity of each multi-item scale using confirmatory factor analysis (CFA). We assessed the results of the CFA from the following criteria: root mean square error of approximation (RMSEA)<0.05 indicated a good fit to data and 0.05<RMSEA<0.08 indicated a satisfactory fit (55). RMSEA is most appropriately used in models with many degrees of freedom (56). Comparative fit index (CFI)≥0.95 and standardized root mean square residual (SRMR)<0.09 indicated a good fit to data (55). In the CFA, we first assessed the factorial validity of the scales within each of the 14 job groups. In these analyses, we had few degrees of freedom and therefore assessed model fit using the CFI-and SRMR-coefficients. The next step was to test a hypothesis of factorial invariance of the scales in multi-group CFA. In these analyses, we assessed whether the factor loadings of the items on the latent variable were identical (ie, invariant) across job groups (57). In these analyses, we assessed model fit from the RMSEA-and CFI-coefficients. Finally, using multi-group multi-factor CFA, we tested whether the multi-item scales within each overall domain were empirically distinct. In these analyses, we compare the fit of two models. In the first model we let all items that made up a scale within a given domain load onto a single latent variable. In the second model we let the same items load onto the latent variables (dimensions) that make up a given domain. For each of the 26 dimensions measured by multiitem scales with ≥3 items, our basic hypotheses regarding factorial validity were that (i) items would load on the latent factor for that dimension and not cross-load on other factors, (ii) the latent factor would explain all correlations between items within that dimension, and (iii) factorial validity would hold within each of the 14 job groups, implying that the grouping of items into scales would be valid within each of the 14 job groups. With regards to factorial invariance, we hypothesized that item thresholds and item loadings would be invariant across job groups, implying that job groups can be compared solely based on their overall score for each dimension. Finally, regarding construct validity, we hypothesized that each scale would show differences between job groups and that the variation between job groups would be largest for scales assessing job-related domains, such as demands at work and work organization and job content, while job group-variation would be smaller for the domain on interpersonal relations.
Our aim was to construct multi-item scales with three or four items. Decisions to omit items from specific multi-item scales were made on the basis of confirmatory factor analyses and analyses of internal consistency reliabilities. Items exhibiting low correlations (<0.4) with the other items in specific multi-item scales across the 14 job groups were omitted. If two items exhibited very high inter-correlations (>0.7), one of the two items were dropped on the basis of an analysis of the content of the questions posed in the items.
Reliabilities and means were assessed using SAS 9.4 (SAS Institute Inc, Cary, NC, USA) and the confirmatory factor analyses were conducted using Mplus 7.

Results
The final version of DPQ consisted of 119 items that operationalized 38 dimensions of the psychosocial work environment (28 multi-item scales and 10 single item measures) covering five domains of the psychosocial work environment: (i) demands at work (six multi-item scales); (ii) work organization and job content (8 multiitem scales); (iii) interpersonal relations: cooperation and leadership (9 multi-item scales and 1 single item); (iv) conflicts in the workplace (6 single items); and (v) reactions to the work situation (5 multi-item scales and 3 single items).   Table 4 shows internal consistency reliabilities for the 26 multi-item scales with ≥3 items. All multi-item scales exhibit satisfactory reliabilities when analyzed on the entire study population. Table 4 also shows range in Cronbach's α-values for each scale for all 14 job groups. Of the 26 relevant scales, 21 revealed satisfactory α-values for all job groups. The scales cognitive demands, work without boundaries, influence on working hours, predictability, and job insecurity, however, had job group-specific α-values below the 0.7 threshold (e-Appendix 7, www.sjweh.fi/show_abstract. php?abstract_id=3793). Finally, table 4 shows that the difference in means across the 14 job groups differs greatly across the 32 relevant dimensions (see e-Appendix 8 for job groupspecific mean values, www.sjweh.fi/show_abstract. php?abstract_id=3793). The difference in observed mean values varied between 10.0-59.1 indicating that some dimensions showed large differences in the psychosocial work environment of the 14 job groups, whereas the differences were smaller for other dimensions. Table 5 shows results from a series of confirma-tory factor analyses (CFA) of the factorial validity of each multi-item scale in DPQ. It was possible to conduct CFA's within each of the 14 job groups for scales with ≥4 items. Of the 22 scales with ≥4 items, 20 scales exhibited satisfactory factorial validity in all 14 job groups (ie, CFI≥0.95 and SRMR<0.09). In 2 scales (emotional demands and social support from colleagues), we found satisfactory factorial validity within 13 and 11 of the 14 job groups, respectively (e-Appendix 9, add URL). Overall, these results, therefore, support the factorial validity of multi-item scales within each of the 14 job groups.

Reliability and validity of the DPQ
In the analyses, we also investigated whether factor loadings were factorially invariant -ie, whether we could assume identical factor loadings across job groups for each of the 28 multi-item scales. Table 5 shows that the hypothesis of factorial invariance across job groups was supported for 10 of the 28 scales where we could perform this type of analysis (ie, RMSEA<0.08 and CFI≥0.95). Accordingly, the assumption of factorial invariance was not supported in 18 of the 28 multi-item scales.
Finally, we investigated whether the dimensions within each of the four overall domains containing multi-item scales were empirically distinct. Table 6 shows that, for all four domains, the multi-factor model had a significantly better model fit than the one-factor model. For two domains, interpersonal relations: cooperation and leadership and reactions to the work

Discussion
The final version of the DPQ operationalized 38 dimensions of the psychosocial work environment grouped in five overall domains: (i) demands at work, (ii) work organization and job content, (iii) interpersonal relations: cooperation and leadership, (iv) conflicts in the workplace, and (v) reactions to the work situation. A main ambition in the development of the DPQ was to develop a generic questionnaire that was directly applicable in all types of jobs. By stratifying the study population in 14 job groups, we were able to test whether the instruments of the questionnaire (multi-item scales and single items) were reliable and valid in job groups that differed in terms of educational attainment and primary job tasks.

Reliability of measures in the DPQ
The internal consistency reliability was satisfactory for all multi-item scales across the 14 job groups, indicating that the reliability of the scales could be reproduced across job groups differing in terms of educational attainment and primary job tasks. However, when we tested internal consistency in each job group separately, we found Cronbach's α-values below the required threshold in some work groups for the dimensions influence on working hours, predictability, cognitive demands, work without boundaries, and job insecurity. These results can be ascribed to differences in job characteristics in different job groups (eg, fixed work time arrangements for mail carriers and health care helpers) and imply that not all multi-item scales may exhibit sufficient reliability in all of the 14 job groups.
The results from the analysis of the test-retest reliability also supported the reliability of most scales. Only two of 32 tests of this type of reliability yielded results below a satisfactory level of test-retest reliability. The scale role conflicts showed a test-retest reliability of 0.65 but an internal consistency reliability of 0.78. The scale demands to conceal feelings showed a test-retest reliability of 0.67. Since the scale has only two items, internal consistency reliability was not calculated.

Validity of multi-item scales in the DPQ
We conducted several tests of the construct validity of the multi-items scales in the DPQ. We examined the differences in the range of mean scores across the 14 job groups. We found large differences between job groups for dimensions, such as influence at work, that we hypothesized to be predominantly influenced by the type of job the individual holds (58). In contrast, for dimensions related to interpersonal relations such as cooperation between colleagues, we found small differences between job groups. These results support our hypotheses on the construct validity of the measures in the DPQ. However, we also found small differences between job groups for some scales relating to work organization, eg, role clarity. In these cases, organizational and interpersonal factors in workplaces may have a larger impact on the psychosocial working conditions than factors related to the job of the respondent. Accordingly, these differences may imply that both "job factors" and "relational factors" are at play in shaping the psychosocial work environment (58). This hypothesis would need to be tested in a study stratified on workplace rather than job type.
We tested the factorial validity of the multi-item scales in the DPQ using confirmatory factor analysis. We found that the factorial validity was satisfactory within each job group for 20 of the 22 scales with ≥4 items.
In these analyses, we tested the factorial structure of 22 scales in 14 job groups, which resulted in 308 tests of factor structures at the job group level. In 304 of these tests, we found a satisfactory model fit and, overall, these results support our basic hypothesis on the factorial validity of the multi-item scales. These findings imply that the factor structure of the multi-items scales developed in the DPQ can be reproduced across job groups that differ in terms of educational attainment and primary work tasks. Other validation studies (34)(35)(36)(37)50) have not investigated the reliability and the validity of the developed instruments at the level of job groups, which may limit the ability of these studies to generalize the applicability of these questionnaires to different job groups. Moreover, in these analyses we found a satisfactory model fit for the multi-item scales in the job groups where we found internal consistency reliabilities below the 0.7 threshold.
These findings support the assumption of generic applicability of DPQ across different types of jobs. However, when we tested for factorial invariance (ie, that the individual items have identical factor loadings on the latent variable) across job groups, only 10 of the 28 investigated scales exhibited factorial invariance across job groups, thereby partially supporting the basic hypothesis on factorial invariance of multi-item scales. This implies that while the overall factor structure was similar, the factor loadings differed between job groups in 18 of the 28 multi-item scales, suggesting that the relative importance of the individual items may vary from job group to job group. While a few previous studies have reported similar results for single scales (59,60), no previous study has to our knowledge systematically evaluated cross-job comparability of psychosocial work environment scales. Indeed, the ability to systematically evaluate factorial equivalence stems from our decision to sample from 14 distinct job groups. Lack of factorial invariance implies that other characteristics (eg, job group) than the persons' individual exposure may influence differences in measurement (57). Accordingly, we assume that the lack of factorial equivalence reflects the very different content of each job, which again implies that the different items may have different importance in different types of jobs.
Results furthermore showed that the domain interpersonal relations had the largest number of scales satisfying the assumption of factorial invariance, while the domain demands at work had the lowest number of scales satisfying this assumption. This implies that different job groups may have more differing perceptions and understandings of items that operationalize demands at work, whereas the understanding of items operationalizing interpersonal relations in the workplace may be more similar across job groups.
Finally, the results supported the hypothesis that the scales of the DPQ were empirically distinct within each domain of the psychosocial work environment.

Methodological considerations
Another approach to the analysis of the validity of the DPQ could have been to conduct a confirmatory factor analysis for the entire questionnaire and then analyze individual dimensions subsequently (50,61). In the present study, we decided for several reasons to use the individual dimensions as the starting point of our analyses. First, in developing the questionnaire, each item was apportioned to specific dimensions and by starting our analysis by analyzing individual dimensions we were able to test the reliability and validity of each multi-item scale. Second, analyzing all items in one model may on the one side challenge the stability of the test and on the other side make it difficult to identify multi-item scales with poor model fit within the total model. On the dimension level, we added the new dimensions work without boundaries, influence on working hours, possibilities for performing work tasks, unnecessary work tasks, cooperation between colleagues within teams, departments, or groups, cooperation with immediate supervisor, involvement of employees, changes in the workplace, discrimination, harassment (by customers, clients, patients, pupils or relatives with response options for distinguishing whether the harassment has occurred at the workplace or outside the workplace, including in social or electronic media), work engagement, a global question on self-reported stress (with a follow-up question asking whether the source of stress was work, private life or both) and overall assessment of the psychosocial work environment (see also table 4). We removed the COPSOQ-II dimensions of variation at work, social community at work, trust regarding management, familywork conflict, focial inclusiveness, unpleasant teasing, conflicts and quarrels, and gossip and slander. Unlike the COPSOQ-II, the DPQ does not include a fixed set of measures of self-reported health conditions (eg, sleeping troubles, burnout or depressive symptoms). Instead, we recommend that researchers use validated measures from instruments that were designed with the specific aim to measure self-reported health conditions.

Comparison of DPQ with COPSOQ-II and COPSOQ-III
On the item level, we kept 19 items and modified 17 items from COPSOQ-II.
We published the Danish language version of the DPQ on NRCWE's homepage in July 2017. One year later, in July 2018, the COPSOQ international network published the updated COPSOQ-III questionnaire on the network's homepage (www.copsoq-network. org). Compared to the DPQ, the COPSOQ-III made fewer changes to the previous COPSOQ-II. Some of the dimensions added to the DPQ (eg, on influence on working hours or work engagement) were also added to the COPSOQ-III, albeit with slightly different names of the dimensions and with different items. Other dimensions added to the DPQ (eg, possibilities for performing work tasks, or cooperation between colleagues within teams, departments or groups) do not appear in the COPSOQ-III. Unlike the DPQ, the COPSOQ-III contains only few revised COPSOQ-II items and has, in general, focused on continuity in the selected items. We could not find any reliability and validity tests of the COPSOQ-III and therefore cannot compare the psychometric properties

Limitations and strengths
The study was based on 14 job groups, thus, we could not ascertain the validity and reliability of the DPQ in all types of jobs. Accordingly, the validity and reliability of the questionnaire outside these 14 groups remains unknown.
An alternative to examining specific job groups could have been to test the questionnaire in a representative sample of employed persons in Denmark. Previous studies have shown, however, that associations between psychosocial working conditions and eg, risk of longterm sickness absence varies across job groups and that the overall results from a study population consisting of several job groups may differ considerably from the results of the individual job groups (6,8). By using a stratified sample of employees in 14 different job groups differing in terms of educational attainment and primary work tasks, we were able to test the reliability and the validity of the measures within and across job groups. As the job groups cover a range of different positions in the contemporary labor market, the results support the validity and reliability of the DPQ within job groups with widely differing characteristics.
Only a minority of items in the DPQ (44 items) are identical to items from previously validated questionnaires and this may limit the possibilities for cross-study comparisons. We accepted this limitation to follow our main aim of constructing the best possible questionnaire on the basis of the findings from different phases of the development process. Moreover, by including new dimensions of the psychosocial work environment, the DPQ enables new research possibilities and new avenues of inquiry for workplaces using the DPQ for workplace assessments of the psychosocial work environment.
It is a strength of the DPQ that it was developed on the basis of a process involving a thorough review of relevant literature, two rounds of qualitative interviews, involvement of experts and labor market representatives and the application of a wide range of analytical techniques for determining reliability and validity.

Concluding remarks
The results reported in the present study generally support the validity and the reliability of the DPQ within each of the 14 job groups. We conclude that the DPQ is a generic questionnaire that is applicable within different types of jobs characterized by different levels of educational attainment and different types of primary work tasks. The analyses showed, however, that some scales exhibited unsatisfactory reliabilities for some job groups. This implies that differences in job characteristics may influence the reliability of some multi-item scales measuring these characteristics. Moreover, the results indicate that questions about psychosocial working conditions may be understood differently across job groups, which can have implications for the comparability of questionnaire-based measures of psychosocial working conditions across job groups. This needs to be explored further in future studies.
Testing questionnaire-based instruments is a continuous effort and the psychometric properties of the instruments should be continuously tested in future studies utilizing measures from the DPQ. The development of the DPQ is, therefore, not concluded with the presentation of the results in this study as future uses of the DPQ will yield valuable experiences to be utilized in the continuous efforts to keep the questionnaire up to date.