Review

Scand J Work Environ Health 2019;45(6):560-576    pdf

https://doi.org/10.5271/sjweh.3839 | Issue date:

Effectiveness of occupational e-mental health interventions: a systematic review and meta-analysis of randomized controlled trials

by Phillips EA, Gordeev VS, Schreyögg J

Objective This systematic review and meta-analysis investigated the effectiveness of occupational e-mental health interventions aimed at stress, depression, anxiety, burnout, insomnia, mindfulness, well-being, and alcohol misuse and their potential treatment moderators.

Methods We systematically reviewed randomized control trials published in English using three electronic databases (MEDLINE, PsycINFO, CINAHL) and three register trials. A pooled effect size for each mental health area was calculated using random-effects modelling. For each meta-analysis, we conducted an analysis of potential moderators (ie, type of recruitment, age, gender, initial psychological symptoms, guidance, therapy type, and study quality).

Results In total, 50 studies were included in the systematic review, and 34 studies were included in the meta-analyses. We noted moderate treatment effects on stress (Hedges’g=0.54), insomnia (g=0.70), and burnout (g=0.51) and small treatment effects on depression (g=0.30), anxiety (g=0.34), well-being (g=0.35), and mindfulness (g=0.42). The pooled effect on alcohol intake was small and nonsignificant.

Conclusions Our results suggest that occupational e-mental health interventions are associated with significant health improvements. However, more research is required to understand which factors contribute to the variation in effectiveness of particular interventions depending on the mental health area and characteristics of participants and interventions.

This article refers to the following texts of the Journal: 2016;42(5):382-394  2015;41(2):164-174  2014;40(6):582-596  2014;40(6):557-568

The importance of occupational mental health has grown notably during the last two decades (1). Increased demand for action on mental health promotion in the workplace is supported by evidence that the workplace and its environment have a significant influence on mental health and well-being (2). The workplace, where we spend approximately one-third of our adult lives, can either improve our mental well-being and stability or contribute to the development of mental health problems (3). The same factors that can improve our psychological well-being ‒ such as supportive leaders, cooperative coworkers, interesting work content, good work‒life balance, high status of the occupation, and appropriate salary ‒ may turn to stressors in unfavorable situations, which can create sufficient strain to cause mental illness. Long working hours, low salary, occupational stress, discrimination, harassment, an unhealthy working environment, and many other risk factors that workplaces pose can negatively affect mental health (3).

Apart from the indisputable ethical value of individual well-being, there is a strong link between mental health and labor economics. Mental health conditions and illnesses are a serious cost-driver for healthcare systems globally, and they indirectly burden economies due to productivity losses. Employees’ poor mental health can negatively affect productivity, manifesting in increased absenteeism and presenteeism, higher staff turnover, and reduced performance (4). In knowledge-based economies, employers rely on employees’ learning, creativity, and innovation capacities and are therefore interested not only in preventing mental health illnesses but also in promoting positive mental health (5). Due to their flexible modes of delivery, low barriers to access, and cost-saving potential, e-mental health interventions are promising, particularly in the context of occupational health (6).

E-mental health is part of e-health, which is broadly described as “the use of information and communication technologies (ICT) for health” (7). According to this definition, the term e-mental health can be interpreted as the use of ICT technologies in the field of mental health. E-mental health interventions are often relatively short, involve self-help psychological treatments, and based on established psychotherapy approaches such as cognitive behavioral theory; in addition, in recent years, they have been delivered via ICT, mostly through online platforms accessible via personal computers, tablets, or smartphones (8,9). One of the key components of an e-mental health intervention is the use of personal guidance through human contact and interaction expressed in a variety of forms such as e-mail or SMS reminders, feedback from a personal/online therapist or a coach, and peer/discussion support groups (10). Occupational e-mental health applies to the working population and can be defined as “the application of e-mental health in the specific life domain of work; its aims include improving the quality of working life and protecting and promoting the safety, health, and well-being of workers.” (11)

The question of the effectiveness of e-mental health interventions in general has been addressed in several systematic reviews and meta-analyses. Barak et al (12) found a moderate overall mean effect on mental health (Hedges’ g=0.53, further noted as “g”) for 9,764 patients from the general population, which is comparable to the average effect size of traditional, face-to-face therapy effectiveness of psychotherapeutic interventions. Grist et al (13) showed in his meta-analysis of 49 studies that computerized cognitive behavioral therapy (CBT) has a moderate-to-large effect (g=0.77) on clients with common mental health disorders. E-mental health interventions were found to be effective in reducing stress in a general population with small to moderate effects. In another meta-analysis, Heber et al (14) reported a small effect size for stress symptoms (Cohen’s d=0.43, noted as “d” hereafter). Similar results were found by Jayawardene et al (15) in their meta-analysis of the effectiveness of online mindfulness interventions, who found a small treatment effect (g=0.42), as well as by Spijkerman et al (16), who found a moderate effect size (g=0.51). Moderate-to-large effect sizes (0.66<g<0.83) were also found for internet-based interventions for post-traumatic stress (17). Computer-based treatments for depression were found to be effective with small- [(g=0.29) (16), (d=0.32) (18), (d=0.34) (14), (d=0.41) (19)] to-moderate effects (d=0.56) (20) in several meta-analyses. Smartphone-based e-mental health interventions for depression were found to be effective with a small effect (g=0.38) (21). Several meta-analyses demonstrated large effect sizes for e-mental health interventions addressing anxiety (d=0.96) (18), generalized anxiety disorder (d= -0.91) (22) and social anxiety (d=0.86) (23). In another meta-analysis, Heber et al (14) reported a small treatment effect on anxiety (d=0.32). Moderate effect sizes were also found for computerized CBT for insomnia (24). Overall, evidence suggests that e-mental health interventions are promising for reducing mental health conditions in the general population. There are differences in the effectiveness of e-mental health interventions across mental health conditions, and effects on stress, depression, and different anxiety conditions are particularly well investigated.

The evidence regarding occupational e-mental health is more limited. Kuster et al (25) reviewed two studies with computer-based versus in-person stress management programs. The results of the review were conflicting, and the authors considered the studies to offer low-quality evidence. Stratton et al (26) found a small pooled effect of e-mental health interventions on reducing depression, anxiety and stress conditions during the postintervention (g=0.24) and follow up (g=0.23) periods. The interventions were found to be effective for reducing depression and stress with small effects; however, there was no evidence of their effectiveness in anxiety reduction. Carolan et al (27) included 21 randomized control trials (RCT) in a meta-analysis on psychological well-being consisting of studies on stress, anxiety and depression. The authors found a small, statistically significant posttreatment effect on psychological well-being (g=0.37) compared with the control condition.

Previous systematic reviews on e-mental health interventions for employees have revealed (pooled) effects on stress, depression, and anxiety (26, 27). However, these reviews did not investigate how the effectiveness of e-mental health interventions vary across different mental health areas. Such negative psychological conditions as insomnia, burnout, or alcohol misuse lead to a variety of negative organizational outcomes, including poor productivity and employee performance (28). The question of whether occupational e-mental health interventions can be useful in these mental health domains has neither been addressed nor summarized in any other previous systematic reviews. The aim of this systematic review and meta-analysis was to address this question by broadening the focus of the review to every mental health area of occupational e-mental health interventions through a broad search strategy, performing a meta-analysis for each outcome and considering a broader set of moderators obtained from previous research.

Methods

Eligibility criteria

Studies were included if they reported on (i) RCT evaluating (ii) the treatment efficacy (iii) of ICT-based interventions on (iv) any mental health condition in (v) an employee population for any occupation. The publication language was restricted to English. Our review included only RCT as they provide the highest level of evidence because the random allocation of participants to study groups minimizes the influences of selection bias (29, 30). Studies with other designs would not be fully comparable resulting in biased results of the meta-analysis, which additionally would have been difficult to interpret (30, 31). Following the Cochrane Handbook recommendations, we distinguished between inactive and active control group designs (30). Accordingly, estimates of treatment effectiveness depend on a contrast between experimental conditions and counterfactual conditions that should be comparable (32). Studies with inactive control condition estimate absolute effects, whereas studies with active control designs estimate relative effects, showing whether a certain treatment is more effective than another (33). In our review, we aimed to investigate only the absolute treatment efficacy of the e-mental health interventions. For this purpose, we needed to account for the fact that effect sizes calculated from a mixed control group, containing both inactive and active control groups, may be blurred (32). To provide compelling evidence, we limited our selection to only studies with an inactive control condition (eg, no treatment or a waiting list control) in the meta-analysis. As there were only a few studies with an active control condition per mental health outcome, it was not possible to include them as a separate subgroup. In studies with multiple arms, only the treatment and inactive control groups were included. Another inclusion criterion for the meta-analysis was reporting of validated outcomes with reliable validity tested in previous studies. Studies that used outcomes that did not meet these criteria were included only in the qualitative portion of this review (3436). Articles were excluded from the analysis if they did not provide sufficient data to calculate pre-post effect sizes per condition and the author was unable to provide those data or did not respond to a request for the data (3739).

Information sources

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards (40). Three electronic databases (MEDLINE, PsycINFO, CINAHL) and three register trials (Cochrane Central Register of Controlled Trials, World Health Organization International Clinical Trials Registry Platform, US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov) were searched on 1 December 2017. The search was updated on 23 April 2018. Citations from recent meta-analyses and systematic reviews were also reviewed. No time restrictions were set.

Search strategy

In the first step, we conducted a brief review using the terms e-health and mental health to identify mental health domains in which e-health solutions had been implemented to date. In the second step, we added the terms we identified in the first step to the search strategy: stress, depression, anxiety, burnout, insomnia, mindfulness, well-being, and alcohol misuse. A full description of the search terms, strategy and screening stages can be found in the supplementary material (www.sjweh.fi/show_abstract.php?abstract_id=3839).

Study selection

Abstracts of potential studies were independently assessed by the author (EP) and coauthor (VG). Disagreements were discussed with the senior author (JS) until a consensus was reached.

Data extraction process

For each article, a systematic extraction form was used to collect the following data: (i) general study information, design and setting; (ii) participant characteristics; (iii) type of recruitment; (iv) intervention and control condition characteristics; (v) type of guidance; (vi) outcome measures; (vii) significant effects on mental health outcomes compared with the controls; and (viii) overall study attrition rate. Supplementary table S1 summarizes the characteristics of the extracted articles.

Assessment of methodological quality

The methodological appraisal of each study was independently conducted by two authors (EP, VG) with RoB 2.0, a revised tool used to assess the risk of bias in RCT (41). Uncertainties were resolved through discussions with a third reviewer (JS). The following risk of bias domains were applied: (i) bias arising from the randomization process (selection bias); (ii) bias due to deviations from intended interventions (performance bias); (iii) bias due to missing outcome data (attrition); (iv) bias in measurement of the outcome (detection bias); and (v) bias in selection of the reported result (reporting bias). Following the Cochrane guidelines (41), we considered a study to have an overall “high risk of bias” when at least one of the bias domains was assessed to have a “high risk of bias”. Publication bias was assessed for each model through the examination of funnel plots and Egger’s regression tests (42). In addition, the trim-and-fill method was applied to identify studies responsible for asymmetry (43). Finally, to assess the robustness of a significant result, we calculated Orwin’s fail-safe N, which indicates how many studies with effect size zero could be added to the meta-analysis to reduce the statistical significance. The results are considered robust if the fail-safe N is ≥5n+10, where n is the number of studies (44).

Statistical analyses

Calculation of effect size and statistical analyses. For each study, between-group effect sizes were calculated for each outcome variable. Taking into consideration that our review also included studies with small sample sizes (<20) and accounting for recommendations for meta-analyses (45), Hedges’ g was used to measure effect size at both the primary study and meta-analysis levels. To assess different sample sizes and pretest values, an effect size based on the mean pre-post change in the treatment group minus the mean pre-post change in the control group, divided by the pooled pretest standard deviation, was determined (46)1. Four studies did not provide standard deviations and means (4750). For one study (47), we used the effect size transformation formula and calculated Cohen’s d from the eta-squared value and adjusted it for small sample bias (51). For three studies (48,49,50), we calculated the effect size using the p-values from the intention-to-treat (ITT) analysis. Effect sizes of 0.2≤g≤0.5 were considered small, 0.5≤g≤0.8 were considered medium, and g≥0.8 was considered large (52). A P value<0.05 was considered statistically significant. Positive effect sizes with a 95% confidence interval (95% CI) excluding zero indicated that the treatment condition was superior to the control condition. All meta-analyses were conducted in RStudio 1.1.447 using the package “metafor” (53).

Meta-regression. We used a random-effects (RE) model that accounts for differences in the treatment effects from heterogeneous studies. Eight separate meta-analyses were conducted for each mental health domain. In each analysis, mean effect sizes (Hedges’ g), heterogeneity estimates (Q statistic), and percentage of total variation across studies due to heterogeneity (I²) were computed. We assessed degree of heterogeneity with a statistical significance level of P<0.05 as follows: (i) 0‒40% might not be important; (ii) 30‒60% may represent moderate heterogeneity; (iii) 50‒90% may represent substantial heterogeneity; (iv) 75‒100% equals considerable heterogeneity (54). Based on previous research, we hypothesized and tested whether the initial level of mental condition, guidance, type of recruitment, gender, and age of participants would moderate the treatment effects (10, 16, 20). We distinguished between two types of recruitment: workplace recruitment, when research teams and companies involved in the study cooperated directly, and publicly announced community recruitment for the general working population. Moderator analyses were conducted to examine whether the effect of an intervention could be explained by moderating variables. Potential moderators were selected based on theoretical considerations as well as on the insights gained from previous systematic and meta-analysis reviews. For example, previous research showed that participants’ characteristics, such as gender, age, and education, as well as the initial level of mental condition of the participants; intervention characteristics, such as the existence of human support (guidance) and psychotherapy type; and study quality would moderate outcomes (10, 12, 16, 18, 19, 20, 26, 27, 55). In addition, a type of recruitment was coded based on observed methodological differences due to study settings and theoretical considerations that study settings may influence participants’ motivation and compliance. We distinguished between two types of recruitment: workplace recruitment (when research teams and companies involved in the study cooperated directly) and publicly announced community recruitment for the general working population. Due to a lack of data, heterogeneity of variable coding, as well as overall general dominance of the high education level in a sample, we excluded education from the moderator list. Based on an observation made in previous meta-analyses that the intervention type has a moderating role on the overall efficacy, we accounted for different psychotherapy types applied (16, 26, 56). For the moderator analysis, the Knapp & Hartung adjustment (57) was included to provide a conservative approach to moderation estimation, as the number of studies in the separate meta-analyses was relatively low.

Results

Study selection

We identified 3075 articles, 1633 of which were duplicates (figure 1). In the next step, we excluded 1216 articles that did not meet our eligibility criteria. Of the 226 full-text articles retrieved, 50 met the inclusion criteria and were included in the review, 34 of which were included in the meta-analysis. We excluded 10 studies with an active control condition, 3 because they did not report the validity of the measures used (3436) and 3 because they did not provide sufficient data to calculate the pre-post effect size (3739).

Figure 1

PRISMA flow chart of study selection

SJWEH-45-560-g001.tif

Study characteristics

The majority of the studies were performed in the US (16), with a further ten performed in Germany, seven in Japan, five in the Netherlands, five in the UK, three in Sweden, and one each in Finland, Norway, Australia, and Hong Kong. The first study on computerized therapy was published in 1987 (50), and the majority of the studies (45) were published in the last five years, 2013‒2017.

Participant characteristics

The total population included 15 258 participants. The mean male gender rate was 47.4%, and the mean age was 41.8 years. Nineteen studies were conducted in a population with some symptoms of the treated condition. Most of the participants (68.7%) were highly educated with at least some years of university education. In cases in which recruitment was conducted in the workplace, the research teams cooperated with one company or several companies. It is striking that in studies with workplace recruitment settings, mainly knowledge-intensive occupational sectors were presented: information technology (7 companies), healthcare sector (6), education and research (3), communication and media (3), public sector (3), and consulting and banking (2). Only two studies on alcohol consumption and stress reduction were conducted at worksites (34, 39). Full details of each study are displayed in supplementary table S1.

Interventions

The majority of interventions primarily addressed stress (18), while ten studies addressed depression, seven addressed insomnia and mental well-being, five focused on reducing alcohol consumption, and two studies were dedicated to burnout reduction. Three of the studies evaluated smartphone- or app-based interventions (47, 58). The majority of the studies (34) did not use guidance or personal contact. Studies with guidance provided different types of human support, eg, regular calls by a clinical study officer, feedback from a clinical psychologist on home assignments, regular guidance from trained e-coaches, peer group discussions, and virtual class meetings. The level of guidance for each study is described in table S1. Thirty-one of the studies conducted recruitment in the workplace, while the remaining twenty studies used open community recruitment and announced their studies via mass media or by cooperating with professional labor organizations or health insurance companies. Treatment duration ranged from brief interventions of 30 minutes (35) to 8 weeks (27, 59, 60). Some studies offered access to the intervention for three (6164), four (65) or six months (48) without specifying the intended treatment duration. The majority of e-mental health interventions (N=22) were based on CBT, some studies (N=7) gave normative personalized feedback on a treated mental condition, often as part of a general health check, while other studies used mindfulness training as a key treatment element (N=6), used psychoeducation (N=5), and the remaining studies used cognitive training (N=2), problem-solving training (N=2), positive psychology (N=2), and applications of Lazarus & Folkman’s transactional theory (66) of stress and coping (N=2); each study involved resilience training, career identity training, and acceptance and commitment therapy (ACT). Ten of the studies reported offering some type of reward to enhance adherence and encourage participation, seven of which were from the USA.

Comparison group

Thirty-four studies used a waitlist (or a waitlist with treatment as usual) as a control condition, and ten studies were conducted with an active control (AC) condition. Most of the studies with an AC condition used information about the treated mental issue offered via a website, regular e-mails, or booklets as a control condition.

Outcomes

Fourteen studies did not define a primary outcome. Many studies with a primary outcome for one mental condition, eg, stress, also considered depression, anxiety, or burnout as secondary outcomes. All studies used outcome measures with valid and reliable psychometric properties. The outcome measures for each study are provided in table S1. Three studies (34) (35, 36) were excluded from the meta-analysis because of unknown psychometric properties. For alcohol consumption, a standard alcohol unit (SUA) of 10‒12 grams of pure alcohol was selected as a common measure.

Quality of the studies

The overall evaluation of risk of bias for the included RCT is presented in Table 1. The majority of the studies suffered from high attrition rates (>20%), which is common for e-health interventions (11). We assessed the studies to be at high risk for missing outcome bias when the intervention arms had substantial differences in attrition or when one of the intervention arms had a dropout rate of more than 20%. All the included studies used patient self-reported measures and were therefore highly biased in the measurement of outcomes. Furthermore, only two studies blinded the participants so they were not aware of which type of trial they were involved in (67, 68). Because patients must play an active role in psychotherapeutic self-help interventions, it is difficult for researchers to implement effective blinding. For this reason, Berger (69) suggested not using the terms “controlled” or “evidence-based” for psychotherapy studies and introduced a new category for studies that cannot be double-blinded: “uncontrolled clinical data”. According to this terminology, all the studies included in the current meta-analysis involved uncontrolled clinical data. Despite these two bias sources, which were relevant for almost all the studies, it was possible to identify a subgroup of studies for the moderator analysis that included studies that were judged as low on other bias domains. Of the 51 studies, 10 reported a conflict of interest whereby one of the authors had a financial interest in the tested e-mental health intervention.

Table 1

Risk of bias assessment of the included studies. [1= bias arising from the randomization process; 2=bias due to deviations from the intended interventions; 3=bias due to missing outcome data; 4=bias in measurement of the outcome; 5=bias in selection of the reported result; +=high risk of bias; - = low risk of bias; ?= unclear risk of bias.]

Study 1 2 3 4 5 Study 1 2 3 4 5
Abbott et al, 2009 (81) + - - - + Imamura et al, 2014 (102) + - - - +
Aikens et al, 2014 (82) + - + - + Imamura et al, 2016 (65) + - + - +
Allexandre et al, 2016 (84) + - - - + Jonas et al, 2017 (83) + - + - +
Billings et al, 2008 (61) ? - + - - Ketelaar et al, 2013 (85) + - - - ?
Birney et al, 2016 (47) + - + - ? Khadjesari et al, 2014 (68) + + - + ?
Bolier et al, 2013 (63) + - - - + Lappalainen et al, 2013 (86) + - + - +
Borness et al, 2013 (37) ? - - ? - Ly et al, 2014 (58) + - + - +
Boß et al, 2017 (76) + - - - + Matano et al, 2007 (34) ? - ? - +
Bostock et al, 2016 (60) + - - - + Mori et al, 2014 (87) + - + - +
Brendryen et al, 2017 (48) + - - - + Myers et al, 2017 (88) + ? - ? +
Carolan et al, 2017 (27) + - - - + Neumeier et al, 2017 (89) ? - - - +
Cook et al, 2007 (62) + - + - + Persson et al, 2017 (90) + - + - +
Cook et al, 2015 (64) + - + - + Phillips et al, 2014 (67) + + - - +
Doumas et al, 2008 (35) - - ? - + Querstret et al, 2017 (91) + - + - +
Van Drongelen et al, 2014 (92) + - + - + Rexroth et al, 2017 (93) ? - - - +
Ebert et al, 2014 (78) + - + - + Ruwaard et al, 2007 (94) + - - - +
Ebert et al, 2016a (79) + - + - + Shirotsuki et al, 2017 (95) - - + - +
Ebert et al, 2016b (72) + - + - + Smith, 1987 (50) ? - ? - -
Eisen et al, 2008 (39) ? - - - - Taylor et al, 2017 (96) + - + - +
Feicht et al, 2013 (97) + - - - + Suzuki et al, 2008 (99) ? - - - ?
Geraedts et al, 2014a (98) + - - - + Thiart et al, 2015 (70) + - + - +
Grime et al, 2014 (59) - - - - + Umanodan et al, 2014 (100) ? - + - +
Hasson et al, 2005 (49) + - + - - Wolever et al, 2012 (38) ? - - - +
Heber et al, 2016 (71) + - + - + Yamagishi et al, 2008 (36) ? - - - +
Hirsch et al, 2017 (101) ? - - - + Yuan, 2015 (103) + - ? - ?

Meta-analysis

The pre-post between-group effects for stress, depression, anxiety, burnout, insomnia, well-being, mindfulness, and alcohol consumption for all the studies including outliers are presented in Table 2. Below, the results of the eight meta-analyses are discussed according to outcome measure.

Table 2

Pre-post between-group effects for all outcomes including outliers. [CI=confidence interval.]

Outcome measures Comparisons (N) Hedge’s g 95% CI Q-value I2 (%) Fail-Safe N
Stress 22 0.54 0.35−0.72 145.89 a 84.73 1635
Depression 17 0.30 0.18−0.42 40.10 a 61.67 285
Anxiety 15 0.34 0.18−0.50 51.90 a 71.57 259
Burnout 8 0.51 0.26−0.75 34.48 a 79.16 231
Insomnia 7 0.70 0.25−1.15 48.14a 91.88 116
Well-being 7 0.35 0.25−0.46 8.99 0.01 84
Mindfulness 5 0.42 0.24−0.60 4.03 0.00 40

a P<0.001.

Effects on stress. A pooled analysis of 22 RCT using random effects demonstrated a medium effect for reducing perceived stress, with g=0.54 (95% CI 0.35‒0.72, P<0.0001) (figure 2). However, the level of heterogeneity was high (I2=84.73%). The omission of two outliers (70, 71) decreased the effect to g=0.44 (95% CI 0.29‒0.59, P<0.0001) and slightly reduced the heterogeneity (I2=72.81%). No publication bias was identified.

Figure 2

Forest plot of the effect sizes of e-mental health interventions on stress, RE Model: random effects model, CI: confidence interval

SJWEH-45-560-g002.tif

Effects on depression. For the 17 studies with depression as an outcome, a small significant effect was observed (g=0.30, 95% CI 0.18‒0.42, P<0.0001) (figure 3). The level of heterogeneity was substantial (I2=61.67%), and one outlier was detected (72). After omitting this study from the analysis, the effect size dropped slightly to g=0.27 (95% CI 0.16‒0.38, P<0.0001), and the heterogeneity decreased to a moderate level (I²=50.35%). No publication biases were detected.

Figure 3

Forest plot of the effect sizes of e-mental health interventions on depression.

SJWEH-45-560-g003.tif

Effects on anxiety. A pooled analysis of 15 studies demonstrated a small effect on reducing anxiety (g=0.34; 95% CI 0.18‒0.50, P=0.0001) (figure 4). The level of heterogeneity was high (I²=71.57%), and one outlier was detected (71). After removing the outlier, the effect size dropped to g=0.28 (95% CI 0.14‒0.43, P<0.0001), but the heterogeneity remained in the moderate range (I²=58.11%). No publication bias was detected.

Figure 4

Forest plot of the effect sizes of e-mental health interventions on anxiety

SJWEH-45-560-g004.tif

Effects on burnout A significant moderate effect was found for the eight studies aiming to relieve burnout (g=0.51, 95% CI 0.26‒0.75, P<0.0001) (figure 5). The level of heterogeneity was high (I²=79.16%), and no outliers were detected.

Figure 5

Forest plot of the effect sizes of e-mental health interventions on burnout

SJWEH-45-560-g005.tif

Effects on insomnia. Based on seven studies, a significant moderate effect was found (g=0.70, 95% CI 0.25‒1.15, P=0.0022) (figure 6). The level of heterogeneity was high (I²=91.88%), and one outlier was found (70). After removing the outlier, the effect dropped to g=0.52 (95% CI 0.39‒0.65, P<0.0001), and the heterogeneity decreased substantially (I²=0.01%). Meta-regression analysis did not identify any relationship of potential moderators and effect size with insomnia. No publication biases were identified.

Figure 6

Forest plot of the effect sizes of e-mental health interventions on insomnia

SJWEH-45-560-g006.tif

Effects on mental well-being. The overall mean effect size for seven studies on well-being was g=0.35 (95% CI 0.25‒0.46, P<0.0001) (figure 7). The level of heterogeneity was low (I²=0.01%), and no outliers were detected. Using meta-regression analysis, we found no moderators for the pooled effect size. No publication biases were detected.

Figure 7

Forest plot of the effect sizes of e-mental health interventions on well-being.

SJWEH-45-560-g007.tif

Effects on mindfulness. For mindfulness (five studies), a significant, moderate effect was found (g=0.42, 95% CI 0.24‒0.60, P<0.0001) (figure 8). The level of heterogeneity was low (I²=0.00%). The meta-regression analysis did not reveal any significant moderators of the effect size. No publication biases were found.

Figure 8

Forest plot of the effect sizes of e-mental health interventions on mindfulness

SJWEH-45-560-g008.tif

Effects on alcohol intake. The pooled analysis of two RCT with passive control groups demonstrated a small but nonsignificant effect on reducing alcohol intake, with g=0.13 (95% CI -0.23‒0.48, P=0.488). The level of heterogeneity was low (I2=0%); however, I2 is imprecise and has substantial bias when the number of studies is small (73).

Meta-regression analysis. Based on the results of the meta-regression analysis, we found that workplace recruitment had a significant negative association with the intervention effects in stress (slope=-0.47, 95% CI -0.80‒ -0.13, P=0.01), depression (slope=-0.31, 95% CI 0.01‒0.07, P=0.005), anxiety (slope=-0.33, 95% CI -0.63‒ -0.02, P=0.04), and burnout (slope=-0.56, 95% CI -1.03‒ -0.0, P=0.03). Age had a slightly positive influence on the effect size in e-mental health interventions for depression (slope=0.04, 95% CI 0.01‒0.07, P=0.005). An increased stress level among the participants demonstrated a positive significant association with the treatment effect (slope=0.55, 95% CI 0.23‒0.86, P=0.002). The presence of guidance moderated the treatment effect in stress-reducing interventions (slope=0.39, 95% CI 0.03‒0.75, P=0.04). Problem-solving therapy had a positive significant association with the treatment effect in cases of stress (slope=0.60, 95% CI 0.15‒1.04, P=0.01), depression (slope=0.35, 95% CI 0.13‒0.58, P=0.004), and anxiety (slope=0.40, 95% CI 0.12‒0.70, p=0.01). High study quality had a positive influence on the effect size in cases of depression (slope=0.34, 95% CI 0.15‒0.53, P=0.002), anxiety (slope=0.32, 95% CI 0.07‒0.62, P=0.046), and burnout (slope=0.61, 95% CI 0.39‒0.83, P=0.0005).

Subgroup analysis. Possible moderating effects can also be identified by explorative subgroup analyses (74) Exploratory subgroup analysis results are presented in Table 3. In the subgroups with open recruitment, significantly higher effects were found for stress, depression, anxiety, and burnout. In the subgroup participants >40 years, significantly higher effects were found for stress, depression, anxiety, and burnout. The presence of psychological symptoms significantly increased treatment effects in all mental health conditions. The presence of guidance and a higher quality of studies resulted in significantly higher treatment effects for interventions in all mental health areas (except well-being, where guidance was absent). The problem-solving therapy showed significantly higher treatment effects for stress, depression, anxiety, and burnout. For insomnia, significantly higher effects were achieved with mindfulness-based therapy.

Table 3

Subgroup analyses

Outcome measures Ncomp Hedge’s g 95% CI I² (%)
Stress
 Recruitment type
  Workplace 13 0.32a 0.18-0.47 49.76
  Community 9 0.79a 0.47-1.12 89.13
 Age (years)
  >40 10 0.76a 0.45-1.07 86.12
  <40 6 0.29b 0.08-0.49 63.40
 Initial mental condition
  With 9 0.84a 0.55-1.13 87.49
  Without 13 0.27a 0.15-0.39 24.97
 Guidance
  With 9 0.76a 0.44-1.08 85.83
  Without 13 0.38a 0.19-0.56 74.54
 Therapy type
  CBT 11 0.40b 0.15-0.65 83.38
  Problem-Solving 4 1.00a 0.58-1.42 88.26
  Mindfulness 3 0.64a 0.36-0.92 0.00
  Positive Psychology 1 0.41c 0.02-0.80 0.00
  Others 3 0.19c 0.00-0.38 0.00
 Risk of bias
  High 11 0.39a 0.24-0.54 45.53
  Low 11 0.67a 0.36-0.98 89.91
Depression
 Recruitment type
  Workplace 8 0.11c 0.007-0.22 0.00
  Community 9 0.45a 0.28-0.61 62.48
 Age (years)
  >40 10 0.47a 0.33-0.61 35.48
  <40 5 0.13c 0.02-0.24 0.00
 Initial mental condition
  With 11 0.40a 0.21-0.53 62.33
  Without 6 0.20c 0.03-0.35 41.42
 Guidance
  With 6 0.48a 0.33-0.63 0.00
  Without 11 0.23b 0.08-0.37 65.29
 Therapy type
  CBT 9 0.18c 0.04-0.33 18.40
  Problem-Solving 5 0.54a 0.37-0.70 41.15
  Positive Psychology 1 0.19 -0.00-0.39 0.00
  Others 2 0.13 -0.01-0.26 0.00
  Risk of bias 9 0.15a 0.061-0.24 0.00
  High 7 0.52a 0.32-0.71 59.96
  Low
Anxiety
 Recruitment type
  Workplace 9 0.18 0.04-0.32 31.33
  Community 6 0.52a 0.27-0.76 72.06
 Age (years)
  >40 9 0.41a 0.20-0.62 69.71
  <40 4 0.16 -0.06-0.40 52.19
 Initial mental condition
  With 9 0.42a 0.23-0.61 70.12
  Without 6 0.20 -0.04-0.44 59.13
 Guidance
  With 5 0.48b 0.16-0.80 76.29
  Without 10 0.26 0.10-0.41 56.02
 Therapy type
  CBT 8 0.18b 0.06-0.30 0.00
  Problem Solving 5 0.58a 0.37-0.79 63.81
  Others 2 0.32 -0.50-1.15 79.72
 Risk of bias
  High 9 0.18c 0.04-0.31 30.41
  Low 6 0.50a 0.26-0.76 73.66
Burnout
 Recruitment type
  Workplace 2 0.07 -0.17-0.32 0.00
  Open community 6 0.64a 0.40-0.87 71.17
 Age (years)
  >40 7 0.56a 0.30-0.80 79.78
  <40 1 0.05 -0.47-0.57 -
 Initial mental condition
  With 6 0.60a 0.30-0.88 80.88
  Without 2 0.26c 0.02-0.50 0.00
 Guidance
  With 4 0.69a 0.38-0.99 70.84
  Without 4 0.33 -0.01-0.68 79.08
 Therapy type
  CBT 2 0.38b 0.12-0.63 0.00
  Problem-Solving 4 0.66a 0.28-1.04 87.19
  Mindfulness 2 0.26c 0.02-0.50 0.00
 Risk of bias
  High 4 0.23b 0.07-0.38 0.00
  Low 4 0.84a 0.69-1.0 0.00
Insomia
 Recruitment type
  Workplace 1 0.70a 0.42-0.97 0.00
  Community 6 0.70c 0.16-1.24 92.91
 Age (years)
  >40 5 0.80b 0.21-1.40 94.14
  <40 2 0.47 -0.14-1.07 63.38
 Initial mental condition
  With 5 0.80b 0.22-1.39 94.78
  Without 2 0.45 -0.16-1.06 58.5
 Guidance
  With 3 1.00c 0.00-1.99 96.55
  Without 4 0.53 0.30-0.76 39.98
 Therapy type
  CBT 3 0.95 -0.18-2.08 94.86
  Problem-Solving 3 0.45a 0.29-0.61 0.00
  Mindfulness 1 0.70a 0.30-1.09 0.00
 Risk of bias
  High 2 0.47 -0.14-1.07 63.38
  Low 5 0.80b 0.21-1.40 94.14
Well-being
 Recruitment type
  Workplace 4 0.48b 0.18-0.78 58.91
  Community 3 0.30a 0.16-0.44 0.00
 Age (years)
  >40 3 0.23c 0.05-0.41 0.00
  <40 2 0.53c 0.11-0.95 72.96
 Initial mental condition
  With - - - -
  Without 7 0.35a 0.25-0.46 0.01
 Guidance
  With - - - -
  Without 7 0.35a 0.25-0.46 0.01
 Therapy type
  Positive Psychology 3 0.42b 0.15-0.69 65.23
  Mindfulness 2 0.46 -0.03-0.96 63.02
  Others 2 0.30 0.12-0.49 0.00
 Risk of bias
  High 7 0.35a 0.25-0.46 0.01
  Low - - - -
Mindfulness
 Recruitment type
  Workplace 3 0.49a 0.26-0.72 0.00
  Community 2 0.61 -0.32-1.54 64.00
 Age (years)
  >40 2 0.61 -0.32-1.54 64.00
  <40 2 0.42c 0.10-0.74 0.00
 Initial mental condition
  With - - - -
  Without 5 0.42a 0.24-0.60 0.00
 Guidance
  With 1 0.57a 0.23-0.90 0.00
  Without 4 0.36a 0.16-0.57 0.01
 Therapy type
  Mindfulness 4 0.43a 0.20-0.66 17.14
  Positive Psychology 1 0.44c 0.05-0.83 -
 Risk of bias
  High 2 0.33b 0.10-0.55 0.00
  Low 3 0.56a 0.28-0.84 0.01

a P<0.001;

b P<0.01;

c P<0.05.

Discussion

To our knowledge, this systematic review with meta-analyses is the first that evaluated e-mental health interventions for every mental health condition in an occupational context. We found statistically significant moderate effects on stress, insomnia, and burnout and small treatment effects on depression, anxiety, well-being, and mindfulness compared with control groups with inactive control conditions. The meta-analyses of alcohol intake consisted of only two studies, and the effect on alcohol intake was small and nonsignificant.

From the 22 studies on stress included in the meta-analysis, it is evident that stress is the most widespread and growing concern for organizations and is the most common area of application of occupational e-mental health interventions. The significant moderate effect (g=0.54) on stress reduction is comparable to the findings of previous research conducted in the general population (16). The omission of two outliers (71, 70) decreased the effect (from g=0.54 to g=0.44), in accordance with the results of previous meta-analyses in the general population (14, 15). Community recruitment significantly enhanced the treatment effect (g=0.79), while workplace recruitment decreased the intervention effect. The positive treatment effect was significantly increased to a large effect size by the presence of guidance (g=0.76) and initial stress level (g=0.84). Problem-solving therapy also significantly increased the treatment effect (g=1.0).

For depression, we noted a small significant effect (g=0.30) when analyzing 17 studies. After omitting the outlier, the effect size slightly dropped (g=0.27), in accordance with the results of previous research in the general population (14, 16, 18, 19). As in the previous case, the effect size increased to a moderate size (g=0.52) in the subgroup with community recruitment. Higher participant age and better study quality slightly enhanced the effect sizes to g=0.45 and g=0.52, respectively.

In the case of anxiety, we analyzed 15 studies and found a small treatment effect (g=0.34), and the effect slightly dropped after the omission of the outlier (g=0.28). This result corresponds to the results of the meta-analysis in the general population conducted by Heber et al (14), while some previous systematic reviews reported large treatment effects on anxiety (18, 56) in the general population. However, it should be noted that anxiety was not a primary therapeutic target in all 15 interventions included in the meta-analyses on anxiety but was considered to be a secondary outcome in interventions on stress and/or depression. Similar moderating patterns were observed as in previous cases. The effect of the treatment was enhanced by community recruitment (g=0.52), problem-solving therapy (0.58), and the quality of the studies (g=0.50).

For burnout, after analyzing eight studies, we found a moderate effect size (g=0.51) with the same moderators as described earlier, namely, community recruitment (g=0.64) and quality of studies (g=0.84). Our findings are similar to those from the latest RCT on burnout in the general population, in which moderate-to-large effects for burnout were found (75). The meta-analyses of insomnia (seven studies), mental well-being (seven studies), and mindfulness (five studies) showed a moderate effect for insomnia (g=0.70), a small effect for mindfulness (0.42) and a small effect for mental well-being (g=0.35). The effectiveness research on reducing insomnia symptoms in the general population in a previous meta-analysis showed a moderate effect (24). Previous meta-analyses in the general population also demonstrated small effects for well-being and small effects for mindfulness (15, 16). The five studies on alcohol intake reported conflicting results. Two studies on alcohol intake reported small treatment effects (48, 76), while two other studies did not use comparable measures and reported improvements only for binge drinking and consumption reduction (34, 35). The fifth study (68) reported the opposite effect, namely, higher alcohol consumption in the intervention group. The pooled effect of the two studies on alcohol intake included in the meta-analyses was small and nonsignificant. Although current research findings regarding online interventions show improvements in drinking behavior in the general population (77), we recommend a meta-analysis that includes more comparisons to better estimate the effectiveness of e-health interventions on alcohol intake in the occupational context.

From the point of view of study design, we noted that open community recruitment enhances the treatment effect. This finding could be explained by the fact that participants who volunteer for community recruitment have a stronger intrinsic motivation and typically do not experience external expectations, eg, compliance with internal procedures in a workplace environment. Age moderated the effect on depression, and the subgroups with older participants showed significantly higher effects for stress, depression, and burnout. The enhancing role of older participants was previously shown (77) and could be explained by the higher self-disciplined engagement of participants that is required by most self-help interventions. However, this finding should be interpreted with caution, as RCT in younger participants in an occupational context are rare. Thus, we recommend that future research should include different age groups or directly address young employees to close the research gap. The initial level of psychological symptoms moderated the effect on stress, and analyses of subgroups showed the same tendency for all mental health conditions. Healthy populations usually have less room for improvement and, therefore, produce smaller treatment effect. This tendency was also previously reported in reviews on e-health interventions (12, 26).

The fact that guided e-health interventions yield better outcomes was already widely investigated in previous research (10, 18, 20). Our explanatory subgroup analyses also confirmed the advantage of the guidance for all mental health areas. However, the subgroup analyses were underpowered and should be interpreted accordingly.

Problem-solving therapy showed a significant moderating effect on effectiveness in cases of stress, depression, anxiety, and burnout; however, this result should be interpreted cautiously, as four of the six studies involving problem-solving therapy in the whole sample were performed by the same investigation team, which could cause a risk of author bias (71, 72, 78, 79). The subgroup analyses indicated that mindfulness and positive psychology also lead to higher effect sizes, in accordance with findings by other researchers (26). Our findings suggest the need to further investigate the effectiveness of these therapy approaches, as they appear superior to CBT according to our results. The education level of participants was predominantly high, so we could not assess its moderating effect due to homogeneity of the sample. Gender did not show any moderating effects on treatment effect.

Study quality had a moderating effect on depression, anxiety, and burnout. Subgroup analysis showed higher treatment effects in studies with a low bias risk in all mental health areas. A correlation between higher study quality and higher effect size could be explained with a larger range of those studies, careful design including systematic randomization approaches applied as well as low attrition rates that lowered the risk of bias in the quality assessment.

Our meta-analysis has several limitations. Although we identified 35 studies for the meta-analysis, we analyzed the effects per outcome, and some outcome groups consisted of only a few studies (eg, only 2 studies covered alcohol consumption). Some subgroup analyses covered a small number of studies, so it should be acknowledged that these analyses are underpowered, and the findings should therefore be interpreted with caution. While excluding outliers reduced heterogeneity among the studies, I2 remained large for some outcomes, implying a persistent heterogeneity among the studies that cannot be explained by this analysis. All studies were assessed as high risk due to the absence of blinding and self-reported outcomes; this absence is particularly important, as the absence of blinding may cause placebo effects in mental health studies. The majority of studies used a waitlist as a control condition, which may provoke nocebo effects. The status of waiting for a future treatment may weaken patients’ efforts to get better on their own initiative compared with natural conditions. Due to this fact, waitlist design studies can provide inaccurate estimates, resulting in stronger treatment effects (79). Approximately one-third of the articles for some outcomes (eg, insomnia and burnout) were written by the same investigation team, which may cause author bias. High dropout rates (>20%) are another common problem of e-health intervention studies, and this was reinforced by the present review; more than half of the studies (25) suffered from high attrition rates, while five studies did not report on attrition. Thus, it was impossible to appraise the bias associated with missing outcome data. Due to the heterogeneity of the studies, it was not possible to assess the actual amount of treatment and its role on effectiveness, as it was not reported consistently. Certain studies spanned weeks, some offered sessions or modules, and many did not report the extent of the treatment. Finally, the majority of the participants were highly educated at the university level, rendering the results less generalizable for other professional groups.

Based on the results discussed, we can conclude that e-mental health interventions can significantly improve mental health conditions in an occupational context with small to moderate treatment effects. Previous systematic reviews on employee population reported significant mental health improvements with small overall effects for stress, depression, and anxiety (26, 27). Higher moderate treatment effects were achieved for stress, insomnia, and burnout. These findings could potentially be explained by the lower degree of stigmatization linked to stress, sleep problems, or burnout than the stigmatization linked to mental health conditions that are closer to a clinical diagnosis of mental health disorder, such as depression, anxiety and alcoholism. This insight could be important for interventions conducted directly at the workplace. The hypothesis that interventions addressing less stigmatized mental health areas could lead to better treatment effects in an occupational setting should be examined in further studies. This insight could be important for interventions conducted directly at the workplace. Furthermore, the hypothesis that open community recruitment enhances the treatment effect should be tested. As an implication for practitioners, we recommend taking advantage of community recruitment settings, namely, by providing maximum voluntary data security and anonymity and time resources for the treatment for interventions conducted directly at the workplace. From a methodological point of view, we recommend testing the effectiveness of e-mental health interventions using objective outcomes such as biological markers so outcome assessment bias can be avoided. This approach would increase the evidence in this field. The effectiveness of e-mental health interventions should also be tested in less knowledge-based occupational sectors to determine whether occupational e-mental health interventions are effective for workers with lower education levels and for younger people and to examine how such interventions work for a more digital-savvy generation.

References

1 

Riba, MB, Parikh, SV, & Greden, JF. (2019). Mental health in the workplace:Strategies and tools to optimizes outcomes 1st ed. Cham: Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-030-04266-0.

2 

Leka, S, Jain, A, Iavicoli, S, Vartia, M, & Ertel, M. (2011). The role of policy for management of psychosocial risks at the workplace in the European Union. Safety Science, 49(4), 558-564, https://doi.org/10.1016/j.ssci.2010.02.002.

3 

Thomas, JC, Hite, J, Hersen, M, & Thomas, JC (Eds.). (2002). Handbook of mental health in the workplace. Thousand Oaks, London, New Delhi: Sage Publications. Mental health in the workplace:Toward an integration of organizational and clinical theory, research, and practice, pp. 3-15, https://doi.org/10.4135/9781452229386.n1.

4 

Goetzel, RZ, Long, SR, Ozminkowski, RJ, Hawkins, K, Wang, S, & Lynch, W. (2004). Health, absence, disability, and presenteeism cost estimates of certain physical and mental health conditions affecting U.S. employers. J Occup Environ Med, 46(4), 398-412, https://doi.org/10.1097/01.jom.0000121151.40413.bd.

5 

Greden, JF, Garcia-Tosi, R, White Harrington, A, Riba, MB, Parikh, SV, & Greden, JF (Eds.). (2019). Mental health in the workplace:strategies and tools to optimizes outcomes (1st ed). Cham: Springer Nature Switzerland AG. Healthy minds at work:Challenges and strategies for businesses, pp. 1-16, https://doi.org/10.1007/978-3-030-04266-0_1.

6 

Qu, C, Ozersky, S, Riba, MB, Parikh, SV, & Greden, JF (Eds.). (2019). Mental health in the workplace:strategies and tools to optimizes outcomes. 1st ed. Cham: Springer Nature Switzerland AG. Workplace Mental Health in Canada:Towards a digital Future, pp. 27-44, https://doi.org/10.1007/978-3-030-04266-0_3.

7 

World Health Organization. (1996). WHOQOL-BREF:Introduction, administation, scoring and generic version of the assessment:Field trial version. Geneva: WHO.

8 

Christensen, H, & Hickie, IB. (2010). Using e-health applications to deliver new mental health services. Med J Aust, 192(11), 53-56.

9 

Ebert, DD, Erbe, D, Rief, W, & Berking, M (Eds.). (2012). Lehrbuch Psychotherapie. Heidelberg: Springer Verlag. Internetbasierte psychologische Interventionen, pp. 131-140, https://doi.org/10.1007/978-3-642-25523-6_12.

10 

Baumeister, H, Reichler, L, Munzinger, M, & Lin, J. (2014). The impact of guidance on internet-based mental health interventions - a systematic review. Internet Interventions, 1(4), 205-215, https://doi.org/10.1016/j.invent.2014.08.003.

11 

Lehr, D, Geraedts, A, Persson Asplund, R, Khadjesari, Z, Heber, E, de Bloom, J, et al., Cacace, M, Fischer, S, & Wiencke, M (Eds.). (2016). Healthy at work:Interdisciplinary perspectives. Cham: Springer International Publishing. Occupational e-Mental health:current approaches and promising perspectives for promotin mental health in workers, pp. 257-281, https://doi.org/10.1007/978-3-319-32331-2_19.

12 

Barak, A, Liat, H, Boniel-Nissim, M, & Shapira, N. (2008). A comprehensive review and a meta-analysis of the effectivness of internet-based psychotherapeutic interventions. J Tech Hum Serv, 26(2-4), 109-160, https://doi.org/10.1080/15228830802094429.

13 

Grist, R, & Cavanagh, K. (2011). Computerised cognitive behavioural therapy for common mental health disorders, what works, for whom under what circumstances?A systematic review and meta-analysis. J Contemp Psychother, 43(4), 243-251, https://doi.org/10.1007/s10879-013-9243-y.

14 

Heber, E, Ebert, DD, Lehr, D, Cuijpers, P, Berking, M, Nobis, S, et al. (2017). The benefit of web- and computer-based interventions for stress:a systematic review and meta-analysis. J Med Internet Res, 19(2), 32, https://doi.org/10.2196/jmir.5774.

15 

Jayawardene, WP, Lohrmann, DK, Erbe, RG, & Torbi, MR. (2017). Effects of preventive online mindfulness interventions on stress and mindfulness:a meta-analysis of randomized controlled trials. Prev Med Rep, 5, 150-159, https://doi.org/10.1016/j.pmedr.2016.11.013.

16 

Spijkerman, MP, Pots, WT, & Bohlmeijer, ET. (2016). Effectiveness of online mindfulness-based interventions in improving mental health:a review and meta-analysis of randomised controlled trials. Clin Psychol Rev, 45, 102-114, https://doi.org/10.1016/j.cpr.2016.03.009.

17 

Kuester, A, Niemeyer, H, & Knaevelsrud, C. (2016). Internet-based interventions for posttraumatic stress:a meta-analysis of randomised controlled trials. Clin Psychol Rev, 43, 1-16, https://doi.org/10.1016/j.cpr.2015.11.004.

18 

Spek, V, Cuijpers, P, Nyklicek, I, Riper, H, Keyzer, J, & Pop, V. (2007). Internet-based cognitive behaviour therapy for symptoms of depression and anxiety:a meta-analysis. Psychol Med, 37(3), 319-328, https://doi.org/10.1017/S0033291706008944.

19 

Andersson, G, & Cuijpers, P. (2009). Internet-based and other computerized psychological treatments for adult depression:a meta analysis. Cogn Behav Ther, 38(4), 196-205, https://doi.org/10.1080/16506070903318960.

20 

Richards, D, & Richardson, T. (2012). Computer-based psychological treatments for depression:a systematic review and meta-analysis. Clin Psychol Rev, 32(4), 329-342, https://doi.org/10.1016/j.cpr.2012.02.004.

21 

Firth, J, Torous, J, Nicholas, J, Carney, R, Pratap, A, Rosenbaum, S, et al. (2017). The efficacy of smartphone-based mental health interventions for depressive symptoms:a meta-analysis of randomized controlled trials. World Psychiatry, 16(3), 287-298, https://doi.org/10.1002/wps.20472.

22 

Richards, D, Richardson, T, Timulak, L, & McElvaney, J. (2015). The efficacy of internet-delivered treatment for generalized anxiety disorder:a systematic review and meta-analysis. Internet Interventions, 2(3), 272-282, https://doi.org/10.1016/j.invent.2015.07.003.

23 

Tulbure, BT. (2011). The efficacy of internet-supported intervention for social anxiety disorder:a brief meta-analytic review. Procedia Soc Behav Sci, 30, 552-557, https://doi.org/10.1016/j.sbspro.2011.10.108.

24 

Cheng, SK, & Dizon, J. (2012). Computerised cognitive behavioural therapy for insomnia:a systematic review and meta-analysis. Psychother Psychosom, 81(4), 206-216, https://doi.org/10.1159/000335379.

25 

Kuster, AT, Dalsbø, TK, Luong Thanh, BY, Agarwal, A, Durand-Moreau, QV, & Kirkehei, I. (2017). Computer-based versus in-person interventions for preventing and reducing stress in workers. Cochrane Database Syst Rev, 8, https://doi.org/10.1002/14651858.CD011899.pub2.

26 

Stratton, E, Lampit, A, Choi, I, Calvo, RA, Harvey, SB, & Glozier, N. (2017). Effectiveness of e-health interventions for reducing mental health conditions in employees:a systematic review and meta-analysis. PLoS One, 12, 12, https://doi.org/10.1371/journal.pone.0189904.

27 

Carolan, S, Harris, PR, Greenwood, K, & Cavanagh, K. (2017). Increasing engagement with an occupational digital stress management program through the use of an online facilitated discussion group:results of a pilot randomised controlled trial. Internet Interventions, 10, 1-11, https://doi.org/10.1016/j.invent.2017.08.001.

28 

Thomas, JC, & Hersen, M. (2002). Handbook of mental health in the workplace. Thousand Oaks, London, New Dehli: Sage Publications.

29 

Odgaard-Jensen, J, Vist, GE, Timmer, A, Kunz, R, Akl, EA, Schünemann, H, et al. (2011). Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev, 4, https://doi.org/10.1002/14651858.MR000012.pub3.

30 

Higgins, JPT, & Green, S. (2001). Cochrane handbook for systematic reviews of interventions 5.1.0. The Cochrane Collaboration.

31 

Peinemann, F, Tushabe, DA, & Kleijnen, J. (2013). Using multiple types of studies in systematic reviews of health care interventions - a systematic review. PLoS One, 26(8), 85035, https://doi.org/10.1371/journal.pone.0085035.

32 

Karlsson, P, & Bergmark, A. (2015). Compared with what?An analysis of control-group types in Cochrane and Campbell reviews of psychosocial treatment efficacy with substance use disorders. Addiction, 110, 420-428, https://doi.org/10.1111/add.12799.

33 

Makuch, RW, & Johnson, MF. (1989). Dilemmas in the use of active control groups in clinical research. IRB: Ethics & Human Research, 11(1), 1-5, https://doi.org/10.2307/3563823.

34 

Matano, RA, Koopman, C, Wanat, SF, Winzelber, AJ, Whitsell, SD, Westrup, D, et al. (2007). A pilot study of an interactive web site in the workplace for reducing alcohol consumption. J Subst Abuse Treat, 32(1), 71-80, https://doi.org/10.1016/j.jsat.2006.05.020.

35 

Doumas, DM, & Hannah, E. (2008). Preventing high-risk drinking in youth in the workplace:a web-based normative feedback program. J Subst Abuse Treat, 34(3), 263-271, https://doi.org/10.1016/j.jsat.2007.04.006.

36 

Yamagishi, M, Kobayashi, T, & Nakamura, Y. (2008). Effects of web-based career identity training for stress management among Japanese nurses:a randomized control trial. J Occup Health, 50(2), 191-193, https://doi.org/10.1539/joh.L7086.

37 

Borness, C, Proudfoot, J, Crawford, J, & Valenzuela, M. (2013). Putting brain training to test in the workplace:a randomized, blinded multisite, active-controlled trial. PLoS One, 8, 3, https://doi.org/10.1371/journal.pone.0059982.

38 

Wolever, RQ, Bobinet, KJ, McCabe, K, Mackenzie, ER, Fekete, E, Kusnick, CA, et al. (2012). Effective an viable mind-body stress reduction in the workplace:a randomized controlled trial. J Occup Health Psychol, 17(2), 246-258, https://doi.org/10.1037/a0027278.

39 

Eisen, KP, Allen, GJ, Bollash, M, & Pescatello, LS. (2008). Stress management in the workplace:a comparison of a computer-based and an in-person stress-management intervention. Comput Human Behav, 24(2), 486-496, https://doi.org/10.1016/j.chb.2007.02.003.

40 

Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, & The PRISMA, Group. Preferred reporting items for systematic reviews and meta-analyses:The PRISMA Statement. PLoS, 6(7), e1000097, https://doi.org/10.1371/journal.pmed.1000097.

41 

Higgins, JPT, Sterne, JAC, Savović, J, Page, MJ, Hróbjartsson, A, Boutron, I, et al. (2016). Revised tool for assessing risk of bias in randomized trials. Cochrane Database Syst Rev, 10(1), 29-31.

42 

Sterne, J, Egger, M, Suton, AJ, Borenstein, M, & Rothstein, HR (Eds.). (2005). Publication bias in meta-analysis:prevention, assessment, and adjustments. Chichester: John Wiley & Sons. Regression methods to detect publication and other bias in meta-analysis, pp. 99-110, https://doi.org/10.1002/0470870168.ch6.

43 

Duval, S, & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publicaiton in bias in meta-analysis. Biometrics, 56, 455-463, https://doi.org/10.1111/j.0006-341X.2000.00455.x.

44 

Orwin, R. (1983). A fail-safe N for effect size in meta-analysis. J Educ Behav Stat, 8(2), 157-159, https://doi.org/10.2307/1164923.

45 

Elis, P. (2010). The essential guide to effect sizes:Statistical power, meta-analysis, and the interpretation of research results. Cambride: Cambridge University Press.

46 

Morris, SB. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Market Research, 11(2), 364-386, https://doi.org/10.1177/1094428106291059.

47 

Birney, AJ, Gunn, R, Russell, JK, & Ary, DV. (2016). MoodHacker mobile web app with email for adults to self-manage mild-to-moderate depression:Randomized controlled trial. JMIR Mhealth Uhealth, 4(1), e8, https://doi.org/10.2196/mhealth.4231.

48 

Brendryen, H, Johansen, A, Duckert, F, & Nesvag, S. (2017). A pilot randomized controlled trial of an internet-based alcohol intervention in a workplace setting. Int J Behav Med, 24(5), 768-777, https://doi.org/10.1007/s12529-017-9665-0.

49 

Hasson, D, Anderber, UM, Theorell, T, & Arnetz, BB. (2005). Psychophysiological effects of a web-based stress management system:a prospective, randomized controlled intervention of study of IT and media workers. BMC Public Health, 78(5), https://doi.org/10.1186/1471-2458-5-78.

50 

Smith, JJ. (1987). The effectiveness of a computerized self-help stress coping program with adult males. Computers in Human Services, 2(1-2), 37-49, https://doi.org/10.1300/J407v02n01_04.

51 

Hedges, L, & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.

52 

Cohen, J. (1988). Statistical power analysis for the behavioral sciences 2nd ed. Hillsdale, NJ: Erlbaum.

53 

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. J. Stat. Softw, 36(3), 1-48, https://doi.org/10.18637/jss.v036.i03.

54 

Higgins, JPT, Thompson, SG, Deeks, JJ, & Altman, DG. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557-560, https://doi.org/10.1136/bmj.327.7414.557.

55 

Cowpertwait, L, & Clarke, D. (2013). Effectiveness of web-based psychological interventions for depression:a meta-analysis. Int J Ment Health Addict, 11(3), 247-268, https://doi.org/10.1007/s11469-012-9416-z.

56 

Saddichha, S, Al-Desouki, M, Lamia, A, Linden, IA, & Krausz, M. (2014). Online interventions for depression and anxiety - a systematic review. Health Psychol Behav Med, 2(1), 841-881, https://doi.org/10.1080/21642850.2014.945934.

57 

Hartung, J, & Knapp, G. (2001). On test fo the overall treatment effect in meta-analysis with normally distributed responses. Stat med, 20(12), 1771-1782, https://doi.org/10.1002/sim.791.

58 

Ly, KH, Asplund, K, & Andersson, G. (2014). Stress management fo middle managers via an acceptance and commitment-based smartphone application:a randomized controlled trial. Internet interventions, 1(3), 95-101.

59 

Grime, PR. (2004). Computerized cognitive behavioural therapy at work:arandomized controlled trial in employees with recent stress-related absenteeism. Occup Med (Lond), 54(5), 353-359, https://doi.org/10.1093/occmed/kqh077.

60 

Bostock, S, Luik, A, & Espie, CA. (2016). Sleep and productivity benefits of digital cognitive behavioural therapy for insomnia:a randomised controlled trial conducted in the workplace environment. J Occup Environ Med, 58(7), 683-689, https://doi.org/10.1097/JOM.0000000000000778.

61 

Billings, DW, Cook, RF, Hendrickson, A, & Dove, DC. (2008). A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med, 50(8), 960-968, https://doi.org/10.1097/JOM.0b013e31816c435b.

62 

Cook, RF, Billings, DW, Hersch, RK, Back, AS, & Hendrickson, A. (2007). A field test of web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity:randomized controlled trial. J Med Internet Res, 9(2), 1-14, https://doi.org/10.2196/jmir.9.2-ne17.

63 

Bolier, L, Ketelaar, SM, Nieuwenhijsen, K, Smeets, O, Gartner, FR, & Sluiter, JK. (2014). Workplace mental health promotion online to enhance well-being of nurses and allied health professionals:a cluster-randomized controlled trial. Internet Interventions, 1(4), 196-204, https://doi.org/10.1016/j.invent.2014.10.002.

64 

Cook, RF, Hersch, RK, Schlossberg, D, & Leaf, SL. (2015). A web-based health promotion program for older workers:randomized controlled trial. J Med Internet Res, 17(3), e82, https://doi.org/10.2196/jmir.3399.

65 

Imamura, K, Kawakami, N, Tsuno, K, Tsuchiya, M, Shimada, K, & Namba, K. (2016). Effects of web-based stress and depression among workers:a randomized controlled trial. J Affect Disord, 203, 30-37, https://doi.org/10.1016/j.jad.2016.05.045.

66 

Lazarus, RS, & Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer.

67 

Phillips, R, Schneider, J, Molosankwe, I, Leese, M, Foroushani, PS, Grime, P, et al. (2014). Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms:effectiveness and costs fo a workplace intervention. Psychol Med, 44(4), 741-752, https://doi.org/10.1017/S0033291713001323.

68 

Khadjesari, Z, Freemantle, N, Linke, S, Hunter, R, & Murray, E. (2014). Health on the web:randomised controlled trial of online screening and brief alcohol intervention delivered in a workplace setting. PLoS One, 9(11), e112553, https://doi.org/10.1371/journal.pone.0112553.

69 

Berger, D. (2015). Double blinding requirement for validity claims in cognitive-behavioral intervention trials for major depressive disorder Analysis of Hollon S. Effect of cognitive therapy with antidepressant medications vs antidepressants alone on the rate. F1000 Res, 4, 639, https://doi.org/10.12688/f1000research.6954.1.

70 

Thiart, H, Lehr, D, Ebert, DD, Berking, M, & Riper, H. (2015). Log in and breathe out:internet-based recovery training for sleepless employees with work-related strain-results of a randomized controlled trail. Scand J Work Environ Health, 41(2), 164-174, https://doi.org/10.5271/sjweh.3478.

71 

Heber, E, Lehr, D, Ebert, DD, Berking, M, & Riper, H. (2016). Web-Based and mobile stress management interventions for employees:a randomized controlled trial. J Med Internet Res, 18(1), 21, https://doi.org/10.2196/jmir.5112.

72 

Ebert, DD, Heber, E, Berking, M, Riper, H, Cuijpers, P, Funk, B, et al. (2016). Self-guided internet-based and mobile based stress management for employees:results of a randomised controlled trial. Occup Environ Med, 73(5), 315-323, https://doi.org/10.1136/oemed-2015-103269.

73 

Von Hippel, PT. (2015). The heterogeneity statistic I²can be biased in small meta-analyses. BMC Medical Research Methodology, 15, 35, https://doi.org/10.1186/s12874-015-0024-z.

74 

Helm, R, & Mark, A. (2012). Analysis and evaluation of moderator effects in regression models:state of art, alternatives and empirical example. Review of Managerial Science, 6(4), 307-332, https://doi.org/10.1007/s11846-010-0057-y.

75 

Hofer, PD, Waadt, M, Aschwanden, R, Milidou, M, Acker, J, Meyer, AH, et al. (2018). Self-help for stress and burnout without therapist contact:an online randomised controlled trial. Work&Stress, 32(2), 189-208, https://doi.org/10.1080/02678373.2017.1402389.

76 

Boß, L, Lehr, D, Schaub, MP, Castro, RP, Riper, H, Berking, M, et al. (2018). Efficacy of a web-based intervention with and without guidance for employees with risky drinking:results of a three-arm randomized controlled trial. Addiction, 113(4), 635-646, https://doi.org/10.1111/add.14085.

77 

Riper, H, Hoogendoorn, A, Cuijpers, P, Karyotaki, E, Boumparis, N, Mira, A, et al. (2018). Effectiveness and treatment moderators of internet interventions for adult problem drinking:an individual patient data meta-analysis of 19 randomised controlled trials. PLoS Med, 15(12), https://doi.org/10.1371/journal.pmed.1002714.

78 

Ebert, DD, Lehr, D, Boss, L, Riper, H, Cuijpers, P, Andersson, G, et al. (2014). Efficacy of an internet-based problem-solving training for teachers:results of a randomized controlled trial. Scand J Work Environ Health, 40(6), 582-596, https://doi.org/10.5271/sjweh.3449.

79 

Ebert, DD, Lehr, D, Heber, E, Riper, H, Cuijpers, P, & Berking, M. (2016). Internet- and mobile-based stress management for employees with adherence-focused guidance:efficacy and mechanism of change. Scand J Work Environ Health, 42(5), 382-394, https://doi.org/10.5271/sjweh.3573.

80 

Button, KS, & Munafò, MR. (2015). Addressing risk of bias in trials of cognitive behavioral therapy. Shanghai Arch Psychiatry, 27(3), 144-148.

81 

Abbott, JA, Klein, B, Hamilton, C, & Rosenthal, A. (2009). The impact of online resilience training for sales managers on well-being and work performance. E-J Appl Psychol, 5(1), 89-95, https://doi.org/10.7790/ejap.v5i1.145.

82 

Aikens, KA, Astin, J, Pelletier, KR, Levanovich, K, Baase, CM, Park, YY, et al. (2014). Mindfulness goes to work:impact of an online workplace intervention. J Occup Environ Med, 56(7), 721-731, https://doi.org/10.1097/JOM.0000000000000209.

83 

Jonas, B, Leuschner, F, & Tossmann, P. (2017). Efficacy of an internet-based intervention for burnout:a randomized controlled trial in the German working population. Anxiety, stress, and coping, 30(2), 133-144, https://doi.org/10.1080/10615806.2016.1233324.

84 

Allexandre, D, Bernstein, AM, Walker, E, Hunter, J, Roizen, MF, & Morledge, TJ. (2016). A web-based mindfulness stress management program in a corporate call center:a randomized clinical trial to evaluate the edded benefit of onsite group support. J Occup Environ Med, 58(264), 254-264, https://doi.org/10.1097/JOM.0000000000000680.

85 

Ketelaar, SM, Nieuwenhuijsen, K, Gartner, FR, Bolier, L, Smeets, O, & Sluiter, JK. (2013). Effect of an E-mental health approach to workers'health surveillance versus control group on work funtioning of hospital employees:a cluster-RCT. PLoS One, 8(9), e72546, https://doi.org/10.1371/journal.pone.0072546.

86 

Lappalainen, P, Kaipainen, K, Lappalainen, R, Hoffren, H, Myllymaki, T, Kinnunen, ML, et al. (2013). Feasibility of a personal health technology-based psychological intervention for men with stress and mood problems:randomized controlled pilot trial. JMIR Res Protoc, 2(1), e1, https://doi.org/10.2196/resprot.2389.

87 

Mori, M, Tajima, M, Kimura, R, Sasaki, N, Somemura, H, Ito, Y, et al. (2014). A web-based training program using cognitive behavioral therapy to alleviate psychological distress among employees:randomized controlled pilot trial. JMIR Res Protoc, 3(4), e70, https://doi.org/10.2196/resprot.3629.

88 

Myers, ND, Prilleltensky, I, Prilleltensky, O, McMahon, A, Dietz, S, & Rubenstein, CL. (2017). Efficacy of the fun for wellness online intervention to promote multidimensional well-being:a randomized controlled trial. Prev Sci, 8, 984-994, https://doi.org/10.1007/s11121-017-0779-z.

89 

Neumeier, LM, Brook, L, Dirchburn, G, & Sckopke, P. (2017). Delivering your daily dose of well-being to the workplace:a randomized controlled trial of an online well-being programme for employees. J. Work Organ. Psychol, 26(4), 555-573, https://doi.org/10.1080/1359432X.2017.1320281.

90 

Persson, Asplund R, Dagoo, J, Fjellstrom, I, Niemi, L, Hansson, K, Zeraati, F, et al. (2018). Internet-based stress management for distressed managers:results from a randomised controlled trial. Occup Environ Med, 75(2), 105-113, https://doi.org/10.1136/oemed-2017-104458.

91 

Querstret, D, Cropley, M, & Fife-Schaw, C. (2017). Internet-based instructor-led mindfulness for work-related rumination, fatigue, and sleep:assessing facets of mindulness as mechanisms for change. A randomized waitlist control trial. J Occup Health Psychol, 22(2), 153-169, https://doi.org/10.1037/ocp0000028.

92 

Van Drongelen, A, Boot, CR, Hlobil, H, Twisk, JW, Smid, T, & Beek, AJ. (2014). Evaluation of an mHealth intervention aiming to improve health-related behavior and sleep and reduce fatigue among airline pilots. Scand J Work Environ Health, 40(6), 557-568, https://doi.org/10.5271/sjweh.3447.

93 

Rexroth, M, Michel, A, & Bosch, C. (2017). Promoting well-being by teaching employees how to segment their life domains:effects of an online-based mindfulness intervention. Z Arb Organ, 61(4), 197-212, https://doi.org/10.1026/0932-4089/a000253.

94 

Ruwaard, J, Lange, A, Bouwman, M, Broeksteeg, J, & Schrieken, B. (2007). E-mailed standardized cognitive behavioural treatment of work-related stress:a randomized controlled trial. Cogn Behav Ther, 36(3), 179-192, https://doi.org/10.1080/16506070701381863.

95 

Shirotsuki, K, Nonaka, Y, Abe, K, Adachi, SI, Adachi, S, Kuboki, T, et al. (2017). The effect for Japanese workers of a self-help computerized cognitive behaviour therapy program with a supplement soft drink. BioPsychoSocial Medicine, 11, 23, https://doi.org/10.1186/s13030-017-0109-5.

96 

Taylor, DJ, Peterson, AL, Pruiksma, KE, Young MCCaughan, S, Nicholson, K, Mintz, J, et al. (2017). Internet and in-person cognitive behavioral therapy for insomnia in military personnel:a randomized clinical trial. Sleep, 40(6), 1, https://doi.org/10.1093/sleep/zsx075.

97 

Feicht, T, Wittmann, M, Jose, G, Mock, A, von Hirschhausen, E, & Esch, T. (2013). Evaluation of a seven-week web-based happiness training to improve psychological well-being, reduce stress, and enhance mindfulness and flourishing: a randomized controlled occupational health study. Evid Based Complement Alternat Med 676953, https://doi.org/10.1155/2013/676953.

98 

Geraedts, AS, Kleiboer, AM, Wiezer Noortje, M, Mechelen, W, & Cuijpers, P. (2014). Short-term effects of a web-based guided self-help intervention for employees with depressive symptoms:randomized controlled trial. J Med Internet Res, 16(5), e51, https://doi.org/10.2196/jmir.3185.

99 

Suzuki, E, Tsuchiya, M, Hirokawa, K, Taniguchi, T, Mitsuhashi, T, & Kawakami, N. (2008). Evaluation of an internet-based self-help program for better quality of sleep among Japanese workers:a randomized controlled trial. J Occup Health, 50(5), 387, https://doi.org/10.1539/joh.L7154.

100 

Umanodan, R, Shimazu, A, Minami, M, & Kawakami, N. (2014). Effects of computer-based stress management training on psychological well-being and work performance in japanese employees:a cluster randomized controlled trial. Ind Health, 52(6), 480-491, https://doi.org/10.2486/indhealth.2013-0209.

101 

Hirsch, A, Luellen, UJ, Holder, JM, Steinberg, G, Dubiel, T, Blazejowskyj, A, et al. (2017). Managing depressive symptoms in the workplace using a web-based self-care tool:a pilot randomized controlled trial. JMIR Res Protoc, 6(4), e51, https://doi.org/10.2196/resprot.7203.

102 

Imamura, K, Kawakami, N, Furukawa, TA, Matsuyama, Y, Shimazu, A, Umanodan, R, et al. (2014). Effects of an internet-based cognitive behavioral therapy (iCBT) program in manga format on improving subthreshold depressive symptoms among healthy workers: a randomized controlled trial. PLoS One, 9(5), e97167, https://doi.org/10.1371/journal.pone.0097167.

103 

Yuan, Q. Evaluating the effectiveness of a psychological capital development program on mental health, engagement and work performance. ProQuest Dissertations & Theses A&I.

104 

Cohen, S, Kamarck, T, & Mermelstein, R. (1983). A global measure of perceived stress. J health Soc Hehav, 24(4), 385-396, https://doi.org/10.2307/2136404.

105 

Maslach, C, & Jackson, SE. (1986). Maslach burnout inventory manual. Palo Alto, CA: Consulting Psychologists Press.

106 

Brown, KQ, & Ryan, RM. (2003). The benefits of being present:Mindfulness and its role in psychological well-being. J Pers Soc Psychol, 84, 822-848, https://doi.org/10.1037/0022-3514.84.4.822.

107 

Hays, RD, Sherbourne, CD, & Mazel, RM. (1993). The RAND 36-Item Health Survey 1.0. Health Econ, 2(3), 217-227, https://doi.org/10.1002/hec.4730020305.

108 

Radloff, LS. (1977). The CES-D scale:A self-report depression scale for research in the general population. Appl Psychol Meas, 1(3), 385-401, https://doi.org/10.1177/014662167700100306.

109 

Beck, AT, Epstein, N, Brown, G, & Steer, RA. (1988). An inventory for measuring clinical anxiety:Psychometric properties. J Consult Clin Psychol, 56, 593-897, https://doi.org/10.1037/0022-006X.56.6.893.

110 

McCorkle, R, Cooley, M, & Shea, J. (1998). A user's manual for the symptom distress scale. Philadelphia: University of Pennsylvania, National Institute of Nursing Research.

111 

Lovibond, SH, & Lovibond, PF. (1995). Manual for the Depression Anxiety Stress Scale (2nd ed). Sydney: Psychology Foundation. https://doi.org/10.1037/t39835-000.

112 

Bastien, CH, Vallières, A, & Morin, CM. (2001). Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med, 4, 297-203, https://doi.org/10.1016/S1389-9457(00)00065-4.

113 

Beck, AT, Steer, RA, Ball, R, & Ranieri, W. (1996). Comparison of Beck Depression Inventories-IA and -II in psychiatric outpatients. J Pers Assess, 67, 588-597, https://doi.org/10.1207/s15327752jpa6703_13.

114 

Wolpe, J. (1969). The practice of behavior therapy. New York: Pergamon Press.

115 

Gärtner, FR, Ketelaar, SM, Smeets, O, Bolier, L, Fischer, E, van Dijk, FJ, et al. (2011). The Mental Vitality @Work study:design of a randomized controlled trial on the effect of a workers'health surveillance mental module for nurses and allied health professionals. BMC Public Health, 11, 290, https://doi.org/10.1186/1471-2458-11-290.

116 

Terluin, B, Van Rhenen, W, Schaufeli, WB, & De Haan, M. (2004). The Four-Dimensional Symptom Questionnaire (4DSQ):measuring distress and other mental health problems in a working population. Work and Stress, 18(3), 187-207, https://doi.org/10.1080/0267837042000297535.

117 

Golderberg, D, & Williams, P. (1988). A user's guide to the General Health Questionnaire. Windsor: NFER-Nelson.

118 

Kessler, RC, Andrews, G, Colpe, LJ, Hiripi, E, Mroczek, DK, Normand, SL, Zaslavsky, AM, et al. (2002). Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med, 32(6), 959-976, https://doi.org/10.1017/S0033291702006074.

119 

Montgomery, SA, & Asberg, M. (1979). A new depression scale designed to be sensitive to change. Br. J. Psychiatry, 134(4), 382-389, https://doi.org/10.1192/bjp.134.4.382.

120 

Buysse, DJ, Reynolds, CF, Monk, TH, Berman, SR, & Kupfer, DJ. (1989). The Pittsburgh Sleep Quality Index:a new instrument for psychiatric practice and research. Psychiatry Res, 28(2), 193-213, https://doi.org/10.1016/0165-1781(89)90047-4.

121 

Osipow, SH, & Spokane, AR. (1983). A manual of occupational stress, strain, and coping. Columbus: Marathon Consulting Press.

122 

Spielberger, CD, Gorssuch, RL, Lushene, PR, Vagg, PR, & Jacobs, GA. (1983). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.

123 

Baer, RA, Smith, GT, Hopkins, J, Krietmeyer, J, & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13(1), 27-45, https://doi.org/10.1177/1073191105283504.

124 

Shimomitsu, T, Yokoyama, K, Ono, Y, Maruta, T, & Tanigawa, T. (1998). Development of a novel brief job stress questionnaire. Report of research grant for the prevention of work related diseases from Labour, 107-15.

125 

Nishikido, N, Kageyama, T, Kobayashi, T, & Haratani, T. (2000). Kani-shitsumonshi niyoru syokugyousei stress no hyoka (Measurements of job stress by a short questionnaire). Occup Mental Health, 8, 73-82.

126 

Levenstein, S, Prantera, C, Varvo, V, Scribano, ML, Berto, E, Luzi, C, et al. (1993). Development of the Perceived Stress Questionnaire:a new tool for psychosomatic research. J Psychosom Res, 37(1), 19-32, https://doi.org/10.1016/0022-3999(93)90120-5.

127 

Spitzer, R, Kroenke, K, & Williams, J. (1999). Validation and utility of a self-report Version of PRIME-MD:the PHQ Primary Care Study. JAMA, 282(18), 1737-1744, https://doi.org/10.1001/jama.282.18.1737.

128 

Geraedts, A, Kleiboer, AM, Twisk, J, Wiezer, Noortje M, Mechelen, W, & Cuijpers, P. (2014). Long-term results of a web-based guided self-help intervention for employees with depressive symptoms:randomized controlled trial. J Med Internet Res, 16(7), 14-28, https://doi.org/10.2196/jmir.3539.

129 

Mundt, JC, Marks, IM, Shear, MK, & Greist, JH. (2002). The Work and Social Adjustment Scale:a simple measure of impairment in functioning. Br. J. Psychiatry, 180, 46-464, https://doi.org/10.1192/bjp.180.5.461.

130 

Evans, C, Connell, J, Barkham, M, Margison, F, McGrath, G, Mellor-Clark, J, et al. (2002). Towards a standardised brief outcome measure:Psychometric properties and utility of the CORE-OM. Br. J. Psychiatry, 180, 51-60, https://doi.org/10.1192/bjp.180.1.51.

131 

Spitzer, RL, Kroenke, K, Williams, JBW, & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder:the GAD-7. Arch Intern med, 166, 1092-1097, https://doi.org/10.1001/archinte.166.10.1092.

132 

McNair, D, Lorr, A, & Droppleman, L. (1971). Edits manual: Profile of mood states. San Diego: Educational and Industrial Testing Services.

133 

Espie, CA, Kyle, SD, Hames, P, et al. (2014). The Sleep Condition Indicator:A clinical screening tool to evaluate insomnia disorder. BMJ Open, 18(4), 3-e004183, https://doi.org/10.1136/bmjopen-2013-004183.

134 

(2018, November 12). World Health Organization, Retrieved from https://www.who.int/ehealth/about/en.

135 

Keyes, CLM. (2002). The mental health continuum:From languishing to flourishing in life. J Health Soc Behav, 43, 207-222, https://doi.org/10.2307/3090197.

136 

Derogatis, LR, & Melisaratos, N. (1983). The Brief Symptom Inventory:An introductory report. Psychol Med, 13(3), 595-605, https://doi.org/10.1017/S0033291700048017.

137 

Flanagan, JC. (1978). A research approach to improving our quality of life. Am Psychol, 33(2), 138-147, https://doi.org/10.1037/0003-066X.33.2.138.

138 

Bech, P, Gudex, C, & Staehr Johansen, K. (1996). The WHO (Ten) well-being index:validation in diabetes. Psychother Psychosom, 65, 183-190, https://doi.org/10.1159/000289073.

139 

Stefano, GB, Benson, H, Fricchione, GL, & Esch, T. (2005). The stress response: Always good and when it is bad. New York: Medical Science International.

140 

Walach, H, Buchheld, N, Buttenmuller, V, Kleinknecht, N, & Schmidt, S. (2006). Measuring mindfulness - The Freiburg Mindfulness Inventory (FMI). Personality and individual differences, 40, 1543-1555, https://doi.org/10.1016/j.paid.2005.11.025.

141 

Prilleltensky, I, Dietz, S, Prillelensky, O, Myers, ND, Rubenstein, CL, Jin, Y, et al. (2015). Assessing multidimensional well-being:Development and validation of the I COPPE scale. Am J Community Psychol, 43(2), 199-226, https://doi.org/10.1002/jcop.21674.

142 

Diener, E, & Ryan, K. (2009). Subjective well-being:a general overview. S Afr J Psychol, 39(4), 391-406, https://doi.org/10.1177/008124630903900402.

143 

Tennant, R, Hiller, L, Fishwick, R, Platt, S, Joseph, S, Weich, S, et al. (2007). The Warwick-Edinburgh mental well-being scale (WEMWBS) development and UK validation. Health Qual Life Outcomes, 5(1), 63, https://doi.org/10.1186/1477-7525-5-63.

144 

Feldman, G, Hayes, A, Kumar, S, Greeson, J, & Laurenceau, JP. (2007). Mindfulness and emotion regulation:the development and initial validaiton of the Cognitive and Affective Mindfulness ScaleRevised (CAMS-R). J Psychopathol Behav Assess, 29(3), 177-190, https://doi.org/10.1007/s10862-006-9035-8.

145 

Demerouti, E, & Nachreiner, F. (1998). Zur Spezifität von Burnout für Dienstleistungsberufe:Fakt oder Artefakt. [The specificity of burnout for social work:fact or artifact] Zeitschrift für Arbeitswissenschaft, 52(2), 82-89.

146 

Orioli, EM, Jaffe, DT, Scott, CD, & Stressmap: Personal diary edition. (1991). The ultimate stress management, self-assessment, and coping guide. New York: Newmarket Press.

147 

Zigmond, AS, & Snaith, RP. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand, 67(6), 361-370, https://doi.org/10.1111/j.1600-0447.1983.tb09716.x.

Notes

1 Morris compared effect size estimates in terms of bias, precision, and robustness to heterogeneity of variance and concluded that pretest standard deviation is a better choice because it represents the standard deviation of the untreated population, which is more likely to be consistent across studies (46).


Additional material