Original article

Scand J Work Environ Health 2016;42(1):17-25    pdf

doi:10.5271/sjweh.3539

Health correlates of workplace bullying: a 3-wave prospective follow-up study

by Bonde JP, Gullander M, Hansen ÅM, Grynderup M, Persson R, Hogh A, Willert MV, Kaerlev L, Rugulies R, Kolstad HA

Objective This study aimed to examine the course of workplace bullying and health correlates among Danish employees across a four-year period.

Methods In total, 7502 public service and private sector employees participated in a 3-wave study from 2006 through 2011. Workplace bullying over the past 6–12 months and data on health characteristics were obtained by self-reports. We identified major depression using Schedules for Clinical Assessment in Neuropsychiatry interviews and the Major Depression Inventory. We performed cross-sectional and longitudinal analyses of outcomes according to self-labelled bullying at baseline using logistic regression.

Results Reports of bullying were persistent across four years in 22.2% (57/257) of employees who initially reported bullying. Baseline associations between self-labelled bullying and sick-listing, poor self-rated health, poor sleep, and depressive symptoms were significant with adjusted odds ratios (OR) ranging from 1.8 [95% confidence interval (95% CI) 1.5–2.4] for poor sleep quality among those bullied “now and then” to 6.9 (95% CI 3.9–12.3) for depression among those reporting being bullied on a daily to monthly basis. In longitudinal analyses adjusting for bullying during follow-up, all health correlates except poor sleep quality persisted up to four years.

Conclusion Self-reported health correlates of workplace bullying including sick-listing, poor self-rated health, depressive symptoms, and a diagnosis of depression tend to persist for several years regardless of whether bullying is discontinued or not. Independent measures of bullying and outcomes are needed to learn whether these findings reflect long lasting health consequences of workplace bullying or whether self-labelled workplace bullying and health complaints are correlated because of common underlying factors.

This article refers to the following texts of the Journal: 2012;38(6):527-536  2012;38(3):218-227

The concept of workplace bullying denotes a situation in which an employee repeatedly and persistently becomes a target of hostile, aggressive, threatening or humiliating behavior from one or more colleagues, managers or clients (1). Often-mentioned examples include direct verbal assault obvious to bystanders, exclusion from social groups, being selectively overlooked in decision-making, being ignored for promotion, and being unreasonably transferred to less qualified work tasks (1).

Although a fair international consensus has been reached about the general concept of bullying, there is no gold standard for assessment in systematic studies (1). Given that researchers use different measures of workplace bullying, it is not surprising that large differences in the prevalence of workplace bullying have been reported over the past 20 years (2). In a review of 86 independent samples including >130 000 employees, the point prevalence of workplace bullying varied 3–24% with a weighted average around 14%, higher in studies using lists of predefined negative acts and lower in studies investigating self-labelled victimization from bullying (2). In addition to factors such as workplace setting, response rates, the definition of workplace bullying and tools used to measure bullying, it is obvious that chosen cut-points for frequency and duration of bullying behavior have a major impact on the reported prevalence.

Numerous cross-sectional and some prospective studies have consistently reported strong associations between self-reported bullying at the workplace and a number of health-related outcomes such as absence due to sickness (3, 4), sleep disorders (5) and common mental health disorders (611). The high prevalence of workplace bullying reported in many studies and the high relative risk (RR) of developing a depressive disorder [RR in the range of 4.2 (6) and 8.5 (7)] has fueled the claim that workplace bullying is the most devastating psychosocial workplace exposure (1). Even if this claim may seem plausible in view of how bullying is defined, it is surprising that – to the best of our knowledge – no published prospective studies are reporting the course over time of self-labelled bullying and correlated symptoms. Hence, it is not known whether a report of being bullied is persistent over several years or just a transient phenomenon. Similarly, it is not known whether poor self-rated health, sleep disturbance, and depressive disorders, which are strongly associated with the perception of being bullied, attenuate across time or whether levels drop to that of non-bullied colleagues when the individual no longer perceives him/or herself as bullied. Answers to these questions may help provide insights into the nature of workplace bullying and assist health professionals in prognostication.

The research questions of this study, therefore, are: (i) How persistent is self-labelled bullying among Danish employees across a 4-year period of time, and is the persistence of bullying modified by a change of job? (ii) How persistent are adverse health correlates of bullying during a 4-year follow-up? The health correlates of bullying included in the analyses are sick leave, poor self-rated health, sleep disturbance, depressive symptoms, and a diagnosis of depression.

Methods

Population

The study population comprised a cohort of public and private sector employees who, in 2006–2007, were enrolled in the Work Bullying and Harassment (WBH)cohort [N=3359, (12)] and PRISME, the Psychosocial Risk Factors for Stress and Mental Disease cohort [N=4489, (13)]. The WBH and PRISME cohorts were established independently for research on health issues related to the psychosocial work environment. The baseline response rate among all those eligible was 45.7% in the WBH cohort and 44.7% in the PRISME cohort. Both cohorts included a baseline questionnaire survey and a follow-up study after two years. After an additional two years, a second follow-up was performed in 2011 for the combined cohort. Loss to follow-up is outlined in table 1. In order to obtain more reliable data on newly-onset major depression, Schedules for Clinical Assessment in Neuropsychiatry (SCAN) interviews were performed in subsets of the PRISME cohort (14) and among employees in the combined cohort selected by screening for depressive symptoms.

Table 1

Response rates by cohort. [WBH=work bullying & harassment; PRISME=psychosocial risk factors for stress & mental disease.]

SJWEH-42-17-g001.tif

Measures of workplace bullying

We used the self-labelling method to measure workplace bullying as proposed by Einarsen and colleagues (15). Identical questions about workplace bullying were asked of both cohorts and at all follow-up rounds. An introductory statement was given to provide questionnaire respondents with the same understanding of bullying: Bullying occurs when, over a longer period of time, one or more persons are repeatedly exposed to unpleasant or negative actions or behaviors at work against which it is difficult to defend oneself. Subsequently, participants were requested to answer the question: “Have you been exposed to bullying at your current workplace within the last 6–12 months?” The question was responded to on a 5-point scale estimating the frequency of bullying: “never”, “now and then”, “monthly”, “weekly”, or “daily”.

Measures of correlates

Social characteristics

Self-reported information on current employment status as of the date of the questionnaire survey was obtained in PRISME and the combined cohort, but this information was not available during follow-up in the WBH cohort. Being sick-listed or unemployed or obtaining other sorts of temporary social allowance, excluding maternity leave, were combined into one dichotomous variable (sick-listed or unemployed; yes/no information on a change of job after baseline was available at the first but not the second follow-up).

Self-rated health

Global self-rated health was measured by a single-item 5-point scale included in the SF-36 questionnaire (16, 17). The question was phrased: “What is your opinion of your overall health?” with the response categories: excellent (0), very good (1), good (2), less than good (3) and bad (4). Poor self-rated health was defined as reporting “less than good” or “bad”.

Sleep quality and disturbed sleep

Sleep quality was measured by a generic question addressing the overall quality of sleep: “How do you rate your overall quality of sleep?” The question was responded to on a 5-point scale: very good (1), good (2), less than good (3), poor (4), and very poor (5). Poor sleep quality was defined as reporting “poor” or “very poor”.

Disturbed sleep was measured by 4 items from a modified version of the Karolinska Sleep Questionnaire [KSQ, (18)]: ie, (i) “how often do you sleep lightly?” (ii) “how often do you have problems falling asleep?” (iii) “how often do you wake up too early and are unable to fall asleep again?” (iv) “how often do you wake up several times and have difficulty falling asleep again?” Participants were asked to focus on the past four weeks. The response categories were: never (0), seldom (1), now and then (2), often (3) and always (4). The 4 item values were averaged to provide a sleep disturbance index (range 0–4), and an average score value >2, representing approximately 15% of the entire population, were categorized as suffering from sleep disturbance.

Depressive symptoms

Depressive symptoms in the PRISME cohort and the combined cohort (second follow-up) were measured by the 6-item subscale of the revised Symptom Check List (SCL90-R) rating scale (19). The depression subscale includes items 15, 22, 26, 29, 30, 79 of the SCL90-R-scale (20) and concern the level of the participants’ distress during the previous four weeks. Each item is rated on a 5-point Likert scale from not at all (0) to extreme (4). The items address a bad mood, a feeling of worthlessness, thoughts about suicide, a feeling of being trapped, feelings of loneliness and self-reproach. The 6-item values were averaged, and an average score >1, corresponding to symptoms being “somewhat or more intense”, was used as the criterion for dichotomizing depressive symptoms.

Depressive symptoms in the WBH cohort were measured by the 12-item version of the Major Depression Inventory (MDI) (21) in which items on being restless and subdued and two items on change of appetite, respectively, were combined. Item response categories ranged from never (0) to all the time (5). Item values were averaged across the 12 items (range 0–5). A score >2 reflects that symptoms were present more than half of the time. This cut-off was the criterion used to dichotomize depressive symptoms, which resulted in approximately the same prevalence of depressive symptoms in the WBH cohort as the PRISME cohort (7–9%).

Diagnosis of depression

We performed standardized psychiatric interviews in a subset of participants from the PRISME and the combined cohort. In addition to the SCL90-DEP scale, we also used data on stress symptoms from the 4-item Perceived Stress Scale-4 (PSS-4) (22) and burnout symptoms from the 6-item Copenhagen Burnout Inventory (CBI) (23) to identify participants eligible for psychiatric interviews. All questions addressed the past four weeks, and responses were given on a 5-point Likert scale (0–5).

Participants were selected for a psychiatric interview if one or more of the following criteria were fulfilled: (i) a score of ≥3 on ≥3 of the 6 items of the SCL90-DEP scale; (ii) a mean score of ≥2.5 on the PSS; and (iii) a mean score of ≥4 on the CBI scale.

In order to identify as many participants with clinical signs of major depression as possible, the selection criteria were defined to obtain the most optimal trade-off between sensitivity and the number of interviews needed. The selection criteria for psychiatric interviews differed slightly between the three waves of examinations.

Ten students of medicine or psychology, trained at a one-week course by a WHO-certified trainer, performed the psychiatric interviews according to SCAN, V.2.1, part I, sections 6, 7, 8 and 10, (24). The Kappa inter-rater reliability coefficient on item level was 0.71 (satisfactory).

All participants fulfilling the diagnostic ICD-10 criteria for a mild, moderate or severe depressive episode in the past three months were categorized as clinically depressed, and all other participants – whether interviewed or not – were classified as not having a depression.

Participants of the WBH cohort did not follow this protocol for SCAN interviews since such interviews were not performed in the first two waves. In this cohort, depression was defined by ICD-10 criteria and measured by the MDI.

Statistical analysis

Research question (i): Persistence of self-labelled bullying during follow-up?

The study base for this analysis was restricted to participants who reported bullying at baseline and completed the following two survey rounds with non-missing information on bullying (N=3435). Self-labelled bullying during follow-up was described by computing with 95% confidence intervals (95% CI) the proportion of those bullied at baseline that also reported bullying after two and four years, respectively. The potential modifying effect of changing jobs from baseline to first follow-up was examined by comparing the prevalence of reporting bullying at the first follow-up in baseline-bullied employees with and without a change of job between baseline and first follow-up. Information on job changes after the first follow-up was not available.

Research question (ii): Persistence of health correlates during follow-up?

This was examined by cross-sectional as well as longitudinal analyses of the entire dataset of 7502 employees and 9644 follow-up rounds with non-missing information on bullying and adverse health.

In cross-sectional analyses

we computed the point prevalence of the adverse health outcomes (current sick-leave or unemployment, poor self-rated health, poor sleep quality and sleep disturbance, depressive symptoms and a diagnosis of depression) at baseline and at each follow-up round according to the baseline prevalence of workplace bullying. Workplace bullying at baseline was categorized as never, now and then, and daily to monthly with the responses daily, weekly, and monthly collapsed into one category because the numbers were small. Odds ratios (OR) and 95% CI were computed by logistic regression. In addition to crude OR, we obtained estimates adjusted for a fixed set of covariates – namely, cohort (WBH/PRISME), gender, age (in years, continuous variable) and length of education after primary school (<3, 3–4, >4 years). The cross-sectional analyses indicate to which degree baseline associations between bullying and adverse health outcomes are attenuated during the subsequent four years. In sensitivity analyses, we restricted the study population to the subset of participants who completed all three rounds and reported no bullying in the second and third round (N=3054) in order to eliminate effects on health correlates of continued reports of bullying.

In longitudinal analyses

we examined whether reports of bullying at baseline predicted an adverse health outcome (yes/no) in one or both follow-up rounds approximately two and four years later. In these models applied to data organized with a survey follow-up round as the observational unit, OR and 95% CI were computed by logistic regression, incorporating a subject-specific random effect to account for the correlation between employees taking part in one or more survey rounds [SAS PROC GLIMMIX procedure, (25)]. In addition to covariates included in cross-sectional models (cohort, gender, age and educational level), the longitudinal risk estimates were also adjusted by survey round (2 and 3, dummy variables) and reports of bullying at one or more follow-up rounds (yes regardless of frequency/no). All analyses were undertaken by SAS software version 9.2 (26).

Results

Non-response

The overall loss of participants at follow-up was 16% at the first follow-up and 40% at the second follow-up with some differences between the cohorts (table 1). Male gender, young age, and less school education was related to higher non-participation in the follow-up study rounds (table 2). Moreover, poor self-rated health, sleep problems and depressive symptoms were reported more frequently at baseline by non-respondents.

Table 2

Loss to follow-up by characteristics at baseline.

SJWEH-42-17-g002.tif

Persistence of self-labelled bullying

The prevalence of self-labelled workplace bullying during follow-up rounds according to bullying at the baseline examination is depicted in table 3. Among the 208 cohort members reporting workplace bullying now and then at baseline (6.1% of 3435 employees completing all three examinations), the prevalence of any frequency in bullying was 28% after approximately two years (59/2008) and 18% (38/208) after approximately four years. Among 49 cohort members who reported daily, weekly, or monthly bullying at baseline (1.4% of all employees completing all three rounds), the prevalence of any bullying at the first and the second follow-up was 41% (20/49) and 38% (19/49), respectively (table 3). Among the 257 employees reporting bullying at baseline, self-labelled bullying at first follow-up was less frequent among employees who discontinued their job after baseline (N=58, 22.7%) in comparison with employees who remained in the same job (N=199, 77.3%, OR 0.29, 95% CI 0.13–0.65).

Table 3

Prevalence of self-labelled workplace bullying at follow-up according to bullying at baseline in the subset of the cohort with complete reporting through all three waves of the survey.

SJWEH-42-17-g003.tif

Persistence of adverse health correlates

The adjusted risk for sick-listing or unemployment, poor self-rated health, poor sleep, depressive symptoms, and a DSM-IV diagnosis of depression was significantly higher among employees with self-labelled bullying compared to employees not reporting bullying (table 4, baseline associations). Associations were stronger among employees reporting bullying frequently (daily to monthly) than those reporting bullying now and then. In additional cross-sectional analyses of each of the two follow-up strata, associations between adverse health outcomes and baseline categories of bullying were weaker, but the adjusted risk was still elevated although not always significantly (table 4, follow-up associations). Sensitivity analyses restricted to participants completing all three rounds (N=3435) and those who did not report bullying at the follow-up rounds (N=3103), respectively, showed essentially the same results (data not shown but available from the first author upon request). Covariates that contributed significantly to the adjusted models included gender, age, and education but not cohort.

Table 4

Cross-sectional analyses: adjusted a odds ratios (ORadj) for sick leave and health outcomes by self-labelled workplace bullying at baseline and two follow-up rounds two years apart.

SJWEH-42-17-g004.tif

In longitudinal analyses, reporting bullying at baseline significantly predicted the examined adverse health outcomes except for poor sleep quality (table 5). In these analyses, effects were adjusted for reports of bullying at follow-up and, therefore, indicate that, except for poor sleep quality, the health correlates of bullying do not decline to reach reference levels of their non-bullied colleagues within a 4-year period.

Table 5

Longitudinal analyses: Adjusted a odds ratios (ORadj) for sick leave and health outcomes through two rounds of follow-up by self-labelled workplace bullying at baseline.

SJWEH-42-17-g005.tif

Discussion

In this large prospective population-based study, we examined how six adverse health characteristics were associated with workplace bullying during a follow-up period of four years. While reporting of workplace bullying was transient in the majority, some 20% among those who initially reported bullied now and then and some 35% of those who initially reported bullied on a daily to monthly basis still reported bullying four years later. As expected in view of previous research, sick leave, poor self-rated health, poor sleep, and depressive symptoms were strongly associated with self-labelled bullying in those reporting bullying now and then and – in particular – among those who reported bullying on a daily to monthly basis. Our interest was to examine whether these strong cross-sectional associations persisted or were attenuated during subsequent years. With few exceptions, cross-sectional associations in each round remained strong during follow-up rounds, and the longitudinal data analyses clearly indicate that all health correlates except for poor sleep quality persisted during several years of follow-up. It is obvious that persisting symptoms in employees who report bullying might be due to continued exposure to bullying during the follow-up period. It is, therefore, noteworthy that only about one-fifth of those reporting bullying at the baseline study still reported bullying four years later and – in particular – that the longitudinal analyses controlling for bullying during follow-up and a sensitivity analysis excluding employees reporting bullying during follow-up yielded results essentially similar to the main analysis.

These findings may be interpreted in several ways. Assuming that health complaints such as poor self-rated health, sleep disturbance and depressive disorder are consequences of being subject to workplace bullying, our findings indicate that, except for sleep problems, these complaints are not very transient but long-lasting and do not reach reference levels within four years even though the majority of employees were no longer bullied during this period. This is compatible with the often-held view that workplace bullying has severe health consequences and, perhaps, represents one of the most serious psychosocial stressors at work (1). Another interpretation is that reports of bullying and health complaints are correlated without being causally related. Since both exposure and outcome are self-reported, associations could simply reflect between-person differences in personality, mood, and attitudes resulting in so-called common method bias (27). For instance, employees with a high level of neuroticism might be more prone either to perceive their environment as aggressive and hostile or to trigger more often aggressive and hostile behavior and, at the same time, they might be more prone to develop depressive symptoms. If so, it would be expected that primary preventive programs aimed at regulating management, social relations and behaviors at work might have an impact on reports of bullying but less so on the health complaints or diseases thought to be caused by bullying. However, it cannot be precluded that a generally friendlier atmosphere at work will affect the reporting of health symptoms in a positive direction simply due to fewer stressors and an increased likelihood of positive outcome expectancies among employees. In any event, from an occupational health prevention point of view, there seems to be a need for exposure measures of workplace bullying that are independent of self-reports to overcome the well-described common method bias (2729). One option is to obtain information from colleagues and managers (30). Finally, it should be kept in mind that variation over time of health indicators such as poor self-rated health, sleep problems, and depressive symptoms are influenced by numerous work and non-work related social and psychological factors other than bullying. Some methodological limitations pertaining to this study need attention. The primary response rate obtained at the baseline survey was <50% in the two cohorts constituting the source populations of this study. Non-response analysis has shown that the non-respondents were more often male and of younger age. Thus, the study population is not entirely representative of all employees. This selection, although speculative, may have an impact on the external validity of this study. More importantly, we have shown that the internal validity in terms of risk of depression according to established risk factors such as age, gender, and socio-economic position in the study population is similar to the risk estimates obtained in the entire population including non-respondents (31). A contributing cause to the low response rate in both cohorts may be that, in addition to completing a questionnaire, respondents were asked to collect and post two or three saliva samples. On the other hand, low response rates were also experienced in other Danish population-based occupational health surveys that did not involve the sampling of biological specimens [for an example, see (32)].

The loss to follow-up among baseline respondents was considerable, reaching almost 40% at the second follow-up after four years. Respondents reporting poorer self-rated health, sleep disturbance and depressive symptoms at baseline were more prevalent among non-respondents during follow-up, which may explain the lower prevalence of most of these outcomes during follow-up compared to baseline. The implication is that the observed attenuation of estimates through follow-up may be biased towards unity (overestimation of the attenuation).

We adjusted both cross-sectional and longitudinal analyses for potential confounding effects of cohort, gender, age and educational level. Studies addressing causal links between self-reported exposures and outcomes often try to counter the risk of common method bias by adjusting for personality traits such as neuroticism (28). However, we don’t think this is an issue in a descriptive study like ours which addressed the time course of perceived bullying and bullying correlates regardless of causality. The prevalence of self-labelled workplace bullying at the 9% observed in this study may be low compared to the overall prevalence estimate of 14% reported in a meta-analysis of 86 independent studies worldwide (2). One explanation is our use of the self-labelled victimization definition of bullying, which results in lower prevalence rates than definitions based on exposure to negative acts at the workplace (2). On the other hand, the prevalence of workplace bullying in our study is not low but actually high compared to an average prevalence estimate of 4.6% (95% CI 3.5–6.1) reported in 13 Scandinavian studies measuring workplace bullying with the self-labelling method that we also applied in our study (2). This disparity may be explained by different reply options. Many earlier studies asked participants to tick yes or no to one item on workplace bullying (1), while we asked about the frequency of bullying during past 6–12 months, including bullying now and then. Fewer than 2% experienced bullying on a monthly, weekly, or daily basis.

Concluding remarks

Self-labelled bullying persists across a four-year follow-up period among 20–40% of a large sample of Danish employees. Sick-leave, poor self-rated health, sleep problems and depressive disorders are strongly associated with reports of workplace bullying. With the exception of poor sleep quality, these bullying health correlates persist over several years regardless of whether bullying is discontinued or not. Independent measures of bullying are needed to learn whether these findings reflect long-lasting health consequences from workplace bullying or whether workplace bullying and health complaints are correlated because of common underlying factors such as personality, mood or attitude.

Acknowledgement

The paper received support from the Danish Work Environment Research Fund (Reference 20100019956/3).

References

1 

Einarsen, S, Hoel, H, Zapf, D, Cooper, CL, Einarsen, S, Hoel, H, Zapf, D, & Cooper, CL (Eds.). (2011). Bullying and Harassment in the workplace: Developments in theory, research and practice. CRC Press: Boca Raton. The concept of bullying and harassment at work: The European tradition.

2 

Nielsen, MB, Mathiessen, SB, & Einarsen, S. (2010). The impact of methodological moderators on prevalence rates of workplace bullying. A meta-analysis. Journal of Occupational and Organizational Psychology, 83, 955-79, http://dx.doi.org/10.1348/096317909X481256.

3 

Kivimaki, M, Elovainio, M, & Vahtera, J. (2000). Workplace bullying and sickness absence in hospital staff. Occup Environ Med, 57, 656-60, http://dx.doi.org/10.1136/oem.57.10.656.

4 

Ortega, A, Christensen, KB, Hogh, A, Rugulies, R, & Borg, V. (2011). One-year prospective study on the effect of workplace bullying on long-term sickness absence. J Nurs Manag, 19, 752-9, http://dx.doi.org/10.1111/j.1365-2834.2010.01179.x.

5 

Niedhammer, I, David, S, Degioanni, S, Drummond, A, Philip, P, Acquarone, D, et al. (2009). Workplace bullying and sleep disturbances: findings from a large scale cross-sectional survey in the French working population. Sleep, 32, 1211-9.

6 

Kivimaki, M, Virtanen, M, Vartia, M, Elovainio, M, Vahtera, J, & Keltikangas-Jarvinen, L. (2003). Workplace bullying and the risk of cardiovascular disease and depression. Occup Environ Med, 60, 779-83, http://dx.doi.org/10.1136/oem.60.10.779.

7 

Rugulies, R, Madsen, IE, Hjarsbech, PU, Hogh, A, Borg, V, Carneiro, IG, et al. (2012). Bullying at work and onset of a major depressive episode among Danish female eldercare workers. Scand J Work Environ Health, 38, 218-27, http://dx.doi.org/10.5271/sjweh.3278.

8 

Nielsen, MB, Mageroy, N, Gjerstad, J, & Einarsen, S. (2014). Workplace bullying and subsequent health problems. Tidsskr Nor Laegeforen, 134, 1233-8, http://dx.doi.org/10.4045/tidsskr.13.0880.

9 

Reknes, I, Pallesen, S, Mageroy, N, Moen, BE, Bjorvatn, B, & Einarsen, S. (2014). Exposure to bullying behaviors as a predictor of mental health problems among Norwegian nurses: results from the prospective SUSSH-survey. Int J Nurs Stud, 51, 479-87, http://dx.doi.org/10.1016/j.ijnurstu.2013.06.017.

10 

Nielsen, MB, & Einarsen, S. (2012). Prospective relationships between workplace sexual harassment and psychological distress. Occup Med (Lond), 62, 226-8, http://dx.doi.org/10.1093/occmed/kqs010.

11 

Einarsen, S, & Nielsen, MB. (2015). Workplace bullying as an antecedent of mental health problems: a five-year prospective and representative study. Int Arch Occup Environ Health, 88, 131-42, http://dx.doi.org/10.1007/s00420-014-0944-7.

12 

Hansen, AM, Hogh, A, Garde, AH, & Persson, R. (2014). Workplace bullying and sleep difficulties: a 2-year follow-up study. Int Arch Occup Environ Health, 87, 285-94, http://dx.doi.org/10.1007/s00420-013-0860-2.

13 

Grynderup, MB, Mors, O, Hansen, AM, Andersen, JH, Bonde, JP, Kaergaard, A, et al. (2012). A two-year follow-up study of risk of depression according to work-unit measures of psychological demands and decision latitude. Scand J Work Environ Health, 38, 527-36, http://dx.doi.org/10.5271/sjweh.3316.

14 

Grynderup, MB, Mors, O, Hansen, AM, Andersen, JH, Bonde, JP, Kaergaard, A, et al. (2013). Work-unit measures of organisational justice and risk of depression--a 2-year cohort study. Occup Environ Med, 70, 380-5, http://dx.doi.org/10.1136/oemed-2012-101000.

15 

Einarsen, S, & Skogstad, A. (1996). Bullying at work: epidemiological findings in public and private organizations. European Journal of Work and Organizational Psychology, 5, 185-202, http://dx.doi.org/10.1080/13594329608414854.

16 

Ware, JE, Jr. (1998). Gandek B. Overview of the SF-36 Health Survey and the International Quality of Life Assessment (IQOLA) Project. J Clin Epidemiol, 51, 903-12, http://dx.doi.org/10.1016/S0895-4356(98)00081-X.

17 

Jylha, M. (2009). What is self-rated health and why does it predict mortality? Towards a unified conceptual model. Soc Sci Med, 69, 307-16, http://dx.doi.org/10.1016/j.socscimed.2009.05.013.

18 

Åkerstedt, T, Hume, K, Minors, D, & Waterhouse, J. (1994). The meaning of good sleep: a longitudinal study of polysomnograph and subjective sleep quality. J Sleep Res, 3, 152-8, http://dx.doi.org/10.1111/j.1365-2869.1994.tb00122.x.

19 

Bech, P, Licht, RW, & Stage, KB. (2005). Rating scales for effective lidelser. Hillerød: Psykiatrisk forksningsenhed.

20 

Pedersen, G, & Karterud, S. (2004). Is SCL-90R helpful for the clinician in assessing DSM-IV symptom disorders? Acta Psychiatr Scand, 110, 215-24, http://dx.doi.org/10.1111/j.1600-0447.2004.00321.x.

21 

Bech, P, Rasmussen, NA, Olsen, LR, Noerholm, V, & Abildgaard, W. (2001). The sensitivity and specificity of the Major Depression Inventory, using the Present State Examination as the index of diagnostic validity. J Affect Disord, 66, 159-64, http://dx.doi.org/10.1016/S0165-0327(00)00309-8.

22 

Cohen, S, Kamarck, T, & Mermelstein, R. (1983). A global measure of perceived stress. J Health Soc Behav, 24, 385-96, http://dx.doi.org/10.2307/2136404.

23 

Kristensen, TS, Borritz, N, Villadsen, E, & Christensen, KB. (2005). The Copenhagen Burnout Inventory: a new tool for the assessment of burnout. Work Stress, 19, 192-207, http://dx.doi.org/10.1080/02678370500297720.

24 

Wing, JK, Sartorius, N, & Ustern, TB. (1998). A reference manual for SCAN. Cambridge: Cambridge university Press. WHO diagnosis and clinical measurement in psychiatry.

25 

SAS Institute Inc SAS STAT 9.2 User’s Guide: The GLIMMIX Procedure (Book Except). (2008). Cary, NC, USA: SAS Institute Inc.

26 

Schabenberger, O, & Wolfinger, R. (2013). SAS/STAT(R) 9.2 User’s Guide. Cary, NC, US: SAS Institute Inc.

27 

Kolstad, HA, Hansen, AM, Kaergaard, A, Thomsen, JF, Kaerlev, L, Mikkelsen, S, et al. (2011). Job strain and the risk of depression: is reporting biased? Am J Epidemiol, 173, 94-102, http://dx.doi.org/10.1093/aje/kwq318.

28 

Zapf, D, Dormann, C, & Frese, M. (1996). Longitudinal studies in organizational stress research: a review of the literature with reference to methodological issues. J Occup Health Psychol, 1, 145-69, http://dx.doi.org/10.1037/1076-8998.1.2.145.

29 

Kasl, SV. (1998). Measuring job stressors and studying the health impact of the work environment: an epidemiologic commentary. J Occup Health Psychol, 3, 390-401, http://dx.doi.org/10.1037/1076-8998.3.4.390.

30 

Gullander, M, Hogh, A, Hansen, AM, Persson, R, Rugulies, R, Kolstad, HA, et al. (2014). Exposure to workplace bullying and risk of depression. J Occup Environ Med, 56, 1258-65, http://dx.doi.org/10.1097/JOM.0000000000000339.

31 

Kaerlev, L, Kolstad, HA, Hansen, AM, Thomsen, JF, Kaergaard, A, Rugulies, R, et al. (2011). Are risk estimates biased in follow-up studies of psychosocial factors with low base-line participation? BMC Public Health, 11, 539, http://dx.doi.org/10.1186/1471-2458-11-539.

32 

Rugulies, R, Aust, B, Burr, H, & Bultmann, U. (2008). Job insecurity, chances on the labour market and decline in self-rated health in a representative sample of the Danish workforce. J Epidemiol Community Health, 62, 245-50, http://dx.doi.org/10.1136/jech.2006.059113.