The importance of occupational mental health has grown notably during the last two decades (1). Increased demand for action on mental health promotion in the workplace is supported by evidence that the workplace and its environment have a significant influence on mental health and well-being (2). The workplace, where we spend approximately one-third of our adult lives, can either improve our mental well-being and stability or contribute to the development of mental health problems (3). The same factors that can improve our psychological well-being ‒ such as supportive leaders, cooperative coworkers, interesting work content, good work‒life balance, high status of the occupation, and appropriate salary ‒ may turn to stressors in unfavorable situations, which can create sufficient strain to cause mental illness. Long working hours, low salary, occupational stress, discrimination, harassment, an unhealthy working environment, and many other risk factors that workplaces pose can negatively affect mental health (3).
Apart from the indisputable ethical value of individual well-being, there is a strong link between mental health and labor economics. Mental health conditions and illnesses are a serious cost-driver for healthcare systems globally, and they indirectly burden economies due to productivity losses. Employees’ poor mental health can negatively affect productivity, manifesting in increased absenteeism and presenteeism, higher staff turnover, and reduced performance (4). In knowledge-based economies, employers rely on employees’ learning, creativity, and innovation capacities and are therefore interested not only in preventing mental health illnesses but also in promoting positive mental health (5). Due to their flexible modes of delivery, low barriers to access, and cost-saving potential, e-mental health interventions are promising, particularly in the context of occupational health (6).
E-mental health is part of e-health, which is broadly described as “the use of information and communication technologies (ICT) for health” (7). According to this definition, the term e-mental health can be interpreted as the use of ICT technologies in the field of mental health. E-mental health interventions are often relatively short, involve self-help psychological treatments, and based on established psychotherapy approaches such as cognitive behavioral theory; in addition, in recent years, they have been delivered via ICT, mostly through online platforms accessible via personal computers, tablets, or smartphones (8,9). One of the key components of an e-mental health intervention is the use of personal guidance through human contact and interaction expressed in a variety of forms such as e-mail or SMS reminders, feedback from a personal/online therapist or a coach, and peer/discussion support groups (10). Occupational e-mental health applies to the working population and can be defined as “the application of e-mental health in the specific life domain of work; its aims include improving the quality of working life and protecting and promoting the safety, health, and well-being of workers.” (11)
The question of the effectiveness of e-mental health interventions in general has been addressed in several systematic reviews and meta-analyses. Barak et al (12) found a moderate overall mean effect on mental health (Hedges’ g=0.53, further noted as “g”) for 9,764 patients from the general population, which is comparable to the average effect size of traditional, face-to-face therapy effectiveness of psychotherapeutic interventions. Grist et al (13) showed in his meta-analysis of 49 studies that computerized cognitive behavioral therapy (CBT) has a moderate-to-large effect (g=0.77) on clients with common mental health disorders. E-mental health interventions were found to be effective in reducing stress in a general population with small to moderate effects. In another meta-analysis, Heber et al (14) reported a small effect size for stress symptoms (Cohen’s d=0.43, noted as “d” hereafter). Similar results were found by Jayawardene et al (15) in their meta-analysis of the effectiveness of online mindfulness interventions, who found a small treatment effect (g=0.42), as well as by Spijkerman et al (16), who found a moderate effect size (g=0.51). Moderate-to-large effect sizes (0.66<g<0.83) were also found for internet-based interventions for post-traumatic stress (17). Computer-based treatments for depression were found to be effective with small- [(g=0.29) (16), (d=0.32) (18), (d=0.34) (14), (d=0.41) (19)] to-moderate effects (d=0.56) (20) in several meta-analyses. Smartphone-based e-mental health interventions for depression were found to be effective with a small effect (g=0.38) (21). Several meta-analyses demonstrated large effect sizes for e-mental health interventions addressing anxiety (d=0.96) (18), generalized anxiety disorder (d= -0.91) (22) and social anxiety (d=0.86) (23). In another meta-analysis, Heber et al (14) reported a small treatment effect on anxiety (d=0.32). Moderate effect sizes were also found for computerized CBT for insomnia (24). Overall, evidence suggests that e-mental health interventions are promising for reducing mental health conditions in the general population. There are differences in the effectiveness of e-mental health interventions across mental health conditions, and effects on stress, depression, and different anxiety conditions are particularly well investigated.
The evidence regarding occupational e-mental health is more limited. Kuster et al (25) reviewed two studies with computer-based versus in-person stress management programs. The results of the review were conflicting, and the authors considered the studies to offer low-quality evidence. Stratton et al (26) found a small pooled effect of e-mental health interventions on reducing depression, anxiety and stress conditions during the postintervention (g=0.24) and follow up (g=0.23) periods. The interventions were found to be effective for reducing depression and stress with small effects; however, there was no evidence of their effectiveness in anxiety reduction. Carolan et al (27) included 21 randomized control trials (RCT) in a meta-analysis on psychological well-being consisting of studies on stress, anxiety and depression. The authors found a small, statistically significant posttreatment effect on psychological well-being (g=0.37) compared with the control condition.
Previous systematic reviews on e-mental health interventions for employees have revealed (pooled) effects on stress, depression, and anxiety (26, 27). However, these reviews did not investigate how the effectiveness of e-mental health interventions vary across different mental health areas. Such negative psychological conditions as insomnia, burnout, or alcohol misuse lead to a variety of negative organizational outcomes, including poor productivity and employee performance (28). The question of whether occupational e-mental health interventions can be useful in these mental health domains has neither been addressed nor summarized in any other previous systematic reviews. The aim of this systematic review and meta-analysis was to address this question by broadening the focus of the review to every mental health area of occupational e-mental health interventions through a broad search strategy, performing a meta-analysis for each outcome and considering a broader set of moderators obtained from previous research.
Methods
Eligibility criteria
Studies were included if they reported on (i) RCT evaluating (ii) the treatment efficacy (iii) of ICT-based interventions on (iv) any mental health condition in (v) an employee population for any occupation. The publication language was restricted to English. Our review included only RCT as they provide the highest level of evidence because the random allocation of participants to study groups minimizes the influences of selection bias (29, 30). Studies with other designs would not be fully comparable resulting in biased results of the meta-analysis, which additionally would have been difficult to interpret (30, 31). Following the Cochrane Handbook recommendations, we distinguished between inactive and active control group designs (30). Accordingly, estimates of treatment effectiveness depend on a contrast between experimental conditions and counterfactual conditions that should be comparable (32). Studies with inactive control condition estimate absolute effects, whereas studies with active control designs estimate relative effects, showing whether a certain treatment is more effective than another (33). In our review, we aimed to investigate only the absolute treatment efficacy of the e-mental health interventions. For this purpose, we needed to account for the fact that effect sizes calculated from a mixed control group, containing both inactive and active control groups, may be blurred (32). To provide compelling evidence, we limited our selection to only studies with an inactive control condition (eg, no treatment or a waiting list control) in the meta-analysis. As there were only a few studies with an active control condition per mental health outcome, it was not possible to include them as a separate subgroup. In studies with multiple arms, only the treatment and inactive control groups were included. Another inclusion criterion for the meta-analysis was reporting of validated outcomes with reliable validity tested in previous studies. Studies that used outcomes that did not meet these criteria were included only in the qualitative portion of this review (34–36). Articles were excluded from the analysis if they did not provide sufficient data to calculate pre-post effect sizes per condition and the author was unable to provide those data or did not respond to a request for the data (37–39).
Information sources
We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards (40). Three electronic databases (MEDLINE, PsycINFO, CINAHL) and three register trials (Cochrane Central Register of Controlled Trials, World Health Organization International Clinical Trials Registry Platform, US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov) were searched on 1 December 2017. The search was updated on 23 April 2018. Citations from recent meta-analyses and systematic reviews were also reviewed. No time restrictions were set.
Search strategy
In the first step, we conducted a brief review using the terms e-health and mental health to identify mental health domains in which e-health solutions had been implemented to date. In the second step, we added the terms we identified in the first step to the search strategy: stress, depression, anxiety, burnout, insomnia, mindfulness, well-being, and alcohol misuse. A full description of the search terms, strategy and screening stages can be found in the supplementary material (www.sjweh.fi/show_abstract.php?abstract_id=3839).
Study selection
Abstracts of potential studies were independently assessed by the author (EP) and coauthor (VG). Disagreements were discussed with the senior author (JS) until a consensus was reached.
Data extraction process
For each article, a systematic extraction form was used to collect the following data: (i) general study information, design and setting; (ii) participant characteristics; (iii) type of recruitment; (iv) intervention and control condition characteristics; (v) type of guidance; (vi) outcome measures; (vii) significant effects on mental health outcomes compared with the controls; and (viii) overall study attrition rate. Supplementary table S1 summarizes the characteristics of the extracted articles.
Assessment of methodological quality
The methodological appraisal of each study was independently conducted by two authors (EP, VG) with RoB 2.0, a revised tool used to assess the risk of bias in RCT (41). Uncertainties were resolved through discussions with a third reviewer (JS). The following risk of bias domains were applied: (i) bias arising from the randomization process (selection bias); (ii) bias due to deviations from intended interventions (performance bias); (iii) bias due to missing outcome data (attrition); (iv) bias in measurement of the outcome (detection bias); and (v) bias in selection of the reported result (reporting bias). Following the Cochrane guidelines (41), we considered a study to have an overall “high risk of bias” when at least one of the bias domains was assessed to have a “high risk of bias”. Publication bias was assessed for each model through the examination of funnel plots and Egger’s regression tests (42). In addition, the trim-and-fill method was applied to identify studies responsible for asymmetry (43). Finally, to assess the robustness of a significant result, we calculated Orwin’s fail-safe N, which indicates how many studies with effect size zero could be added to the meta-analysis to reduce the statistical significance. The results are considered robust if the fail-safe N is ≥5n+10, where n is the number of studies (44).
Statistical analyses
Calculation of effect size and statistical analyses. For each study, between-group effect sizes were calculated for each outcome variable. Taking into consideration that our review also included studies with small sample sizes (<20) and accounting for recommendations for meta-analyses (45), Hedges’ g was used to measure effect size at both the primary study and meta-analysis levels. To assess different sample sizes and pretest values, an effect size based on the mean pre-post change in the treatment group minus the mean pre-post change in the control group, divided by the pooled pretest standard deviation, was determined (46)1. Four studies did not provide standard deviations and means (47–50). For one study (47), we used the effect size transformation formula and calculated Cohen’s d from the eta-squared value and adjusted it for small sample bias (51). For three studies (48,49,50), we calculated the effect size using the p-values from the intention-to-treat (ITT) analysis. Effect sizes of 0.2≤g≤0.5 were considered small, 0.5≤g≤0.8 were considered medium, and g≥0.8 was considered large (52). A P value<0.05 was considered statistically significant. Positive effect sizes with a 95% confidence interval (95% CI) excluding zero indicated that the treatment condition was superior to the control condition. All meta-analyses were conducted in RStudio 1.1.447 using the package “metafor” (53).
Meta-regression. We used a random-effects (RE) model that accounts for differences in the treatment effects from heterogeneous studies. Eight separate meta-analyses were conducted for each mental health domain. In each analysis, mean effect sizes (Hedges’ g), heterogeneity estimates (Q statistic), and percentage of total variation across studies due to heterogeneity (I²) were computed. We assessed degree of heterogeneity with a statistical significance level of P<0.05 as follows: (i) 0‒40% might not be important; (ii) 30‒60% may represent moderate heterogeneity; (iii) 50‒90% may represent substantial heterogeneity; (iv) 75‒100% equals considerable heterogeneity (54). Based on previous research, we hypothesized and tested whether the initial level of mental condition, guidance, type of recruitment, gender, and age of participants would moderate the treatment effects (10, 16, 20). We distinguished between two types of recruitment: workplace recruitment, when research teams and companies involved in the study cooperated directly, and publicly announced community recruitment for the general working population. Moderator analyses were conducted to examine whether the effect of an intervention could be explained by moderating variables. Potential moderators were selected based on theoretical considerations as well as on the insights gained from previous systematic and meta-analysis reviews. For example, previous research showed that participants’ characteristics, such as gender, age, and education, as well as the initial level of mental condition of the participants; intervention characteristics, such as the existence of human support (guidance) and psychotherapy type; and study quality would moderate outcomes (10, 12, 16, 18, 19, 20, 26, 27, 55). In addition, a type of recruitment was coded based on observed methodological differences due to study settings and theoretical considerations that study settings may influence participants’ motivation and compliance. We distinguished between two types of recruitment: workplace recruitment (when research teams and companies involved in the study cooperated directly) and publicly announced community recruitment for the general working population. Due to a lack of data, heterogeneity of variable coding, as well as overall general dominance of the high education level in a sample, we excluded education from the moderator list. Based on an observation made in previous meta-analyses that the intervention type has a moderating role on the overall efficacy, we accounted for different psychotherapy types applied (16, 26, 56). For the moderator analysis, the Knapp & Hartung adjustment (57) was included to provide a conservative approach to moderation estimation, as the number of studies in the separate meta-analyses was relatively low.
Results
Study selection
We identified 3075 articles, 1633 of which were duplicates (figure 1). In the next step, we excluded 1216 articles that did not meet our eligibility criteria. Of the 226 full-text articles retrieved, 50 met the inclusion criteria and were included in the review, 34 of which were included in the meta-analysis. We excluded 10 studies with an active control condition, 3 because they did not report the validity of the measures used (34–36) and 3 because they did not provide sufficient data to calculate the pre-post effect size (37–39).
Study characteristics
The majority of the studies were performed in the US (16), with a further ten performed in Germany, seven in Japan, five in the Netherlands, five in the UK, three in Sweden, and one each in Finland, Norway, Australia, and Hong Kong. The first study on computerized therapy was published in 1987 (50), and the majority of the studies (45) were published in the last five years, 2013‒2017.
Participant characteristics
The total population included 15 258 participants. The mean male gender rate was 47.4%, and the mean age was 41.8 years. Nineteen studies were conducted in a population with some symptoms of the treated condition. Most of the participants (68.7%) were highly educated with at least some years of university education. In cases in which recruitment was conducted in the workplace, the research teams cooperated with one company or several companies. It is striking that in studies with workplace recruitment settings, mainly knowledge-intensive occupational sectors were presented: information technology (7 companies), healthcare sector (6), education and research (3), communication and media (3), public sector (3), and consulting and banking (2). Only two studies on alcohol consumption and stress reduction were conducted at worksites (34, 39). Full details of each study are displayed in supplementary table S1.
Interventions
The majority of interventions primarily addressed stress (18), while ten studies addressed depression, seven addressed insomnia and mental well-being, five focused on reducing alcohol consumption, and two studies were dedicated to burnout reduction. Three of the studies evaluated smartphone- or app-based interventions (47, 58). The majority of the studies (34) did not use guidance or personal contact. Studies with guidance provided different types of human support, eg, regular calls by a clinical study officer, feedback from a clinical psychologist on home assignments, regular guidance from trained e-coaches, peer group discussions, and virtual class meetings. The level of guidance for each study is described in table S1. Thirty-one of the studies conducted recruitment in the workplace, while the remaining twenty studies used open community recruitment and announced their studies via mass media or by cooperating with professional labor organizations or health insurance companies. Treatment duration ranged from brief interventions of 30 minutes (35) to 8 weeks (27, 59, 60). Some studies offered access to the intervention for three (61–64), four (65) or six months (48) without specifying the intended treatment duration. The majority of e-mental health interventions (N=22) were based on CBT, some studies (N=7) gave normative personalized feedback on a treated mental condition, often as part of a general health check, while other studies used mindfulness training as a key treatment element (N=6), used psychoeducation (N=5), and the remaining studies used cognitive training (N=2), problem-solving training (N=2), positive psychology (N=2), and applications of Lazarus & Folkman’s transactional theory (66) of stress and coping (N=2); each study involved resilience training, career identity training, and acceptance and commitment therapy (ACT). Ten of the studies reported offering some type of reward to enhance adherence and encourage participation, seven of which were from the USA.
Comparison group
Thirty-four studies used a waitlist (or a waitlist with treatment as usual) as a control condition, and ten studies were conducted with an active control (AC) condition. Most of the studies with an AC condition used information about the treated mental issue offered via a website, regular e-mails, or booklets as a control condition.
Outcomes
Fourteen studies did not define a primary outcome. Many studies with a primary outcome for one mental condition, eg, stress, also considered depression, anxiety, or burnout as secondary outcomes. All studies used outcome measures with valid and reliable psychometric properties. The outcome measures for each study are provided in table S1. Three studies (34) (35, 36) were excluded from the meta-analysis because of unknown psychometric properties. For alcohol consumption, a standard alcohol unit (SUA) of 10‒12 grams of pure alcohol was selected as a common measure.
Quality of the studies
The overall evaluation of risk of bias for the included RCT is presented in Table 1. The majority of the studies suffered from high attrition rates (>20%), which is common for e-health interventions (11). We assessed the studies to be at high risk for missing outcome bias when the intervention arms had substantial differences in attrition or when one of the intervention arms had a dropout rate of more than 20%. All the included studies used patient self-reported measures and were therefore highly biased in the measurement of outcomes. Furthermore, only two studies blinded the participants so they were not aware of which type of trial they were involved in (67, 68). Because patients must play an active role in psychotherapeutic self-help interventions, it is difficult for researchers to implement effective blinding. For this reason, Berger (69) suggested not using the terms “controlled” or “evidence-based” for psychotherapy studies and introduced a new category for studies that cannot be double-blinded: “uncontrolled clinical data”. According to this terminology, all the studies included in the current meta-analysis involved uncontrolled clinical data. Despite these two bias sources, which were relevant for almost all the studies, it was possible to identify a subgroup of studies for the moderator analysis that included studies that were judged as low on other bias domains. Of the 51 studies, 10 reported a conflict of interest whereby one of the authors had a financial interest in the tested e-mental health intervention.
Table 1
Risk of bias assessment of the included studies. [1= bias arising from the randomization process; 2=bias due to deviations from the intended interventions; 3=bias due to missing outcome data; 4=bias in measurement of the outcome; 5=bias in selection of the reported result; +=high risk of bias; - = low risk of bias; ?= unclear risk of bias.]
| Study | 1 | 2 | 3 | 4 | 5 | Study | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Abbott et al, 2009 (81) | + | - | - | - | + | Imamura et al, 2014 (102) | + | - | - | - | + |
| Aikens et al, 2014 (82) | + | - | + | - | + | Imamura et al, 2016 (65) | + | - | + | - | + |
| Allexandre et al, 2016 (84) | + | - | - | - | + | Jonas et al, 2017 (83) | + | - | + | - | + |
| Billings et al, 2008 (61) | ? | - | + | - | - | Ketelaar et al, 2013 (85) | + | - | - | - | ? |
| Birney et al, 2016 (47) | + | - | + | - | ? | Khadjesari et al, 2014 (68) | + | + | - | + | ? |
| Bolier et al, 2013 (63) | + | - | - | - | + | Lappalainen et al, 2013 (86) | + | - | + | - | + |
| Borness et al, 2013 (37) | ? | - | - | ? | - | Ly et al, 2014 (58) | + | - | + | - | + |
| Boß et al, 2017 (76) | + | - | - | - | + | Matano et al, 2007 (34) | ? | - | ? | - | + |
| Bostock et al, 2016 (60) | + | - | - | - | + | Mori et al, 2014 (87) | + | - | + | - | + |
| Brendryen et al, 2017 (48) | + | - | - | - | + | Myers et al, 2017 (88) | + | ? | - | ? | + |
| Carolan et al, 2017 (27) | + | - | - | - | + | Neumeier et al, 2017 (89) | ? | - | - | - | + |
| Cook et al, 2007 (62) | + | - | + | - | + | Persson et al, 2017 (90) | + | - | + | - | + |
| Cook et al, 2015 (64) | + | - | + | - | + | Phillips et al, 2014 (67) | + | + | - | - | + |
| Doumas et al, 2008 (35) | - | - | ? | - | + | Querstret et al, 2017 (91) | + | - | + | - | + |
| Van Drongelen et al, 2014 (92) | + | - | + | - | + | Rexroth et al, 2017 (93) | ? | - | - | - | + |
| Ebert et al, 2014 (78) | + | - | + | - | + | Ruwaard et al, 2007 (94) | + | - | - | - | + |
| Ebert et al, 2016a (79) | + | - | + | - | + | Shirotsuki et al, 2017 (95) | - | - | + | - | + |
| Ebert et al, 2016b (72) | + | - | + | - | + | Smith, 1987 (50) | ? | - | ? | - | - |
| Eisen et al, 2008 (39) | ? | - | - | - | - | Taylor et al, 2017 (96) | + | - | + | - | + |
| Feicht et al, 2013 (97) | + | - | - | - | + | Suzuki et al, 2008 (99) | ? | - | - | - | ? |
| Geraedts et al, 2014a (98) | + | - | - | - | + | Thiart et al, 2015 (70) | + | - | + | - | + |
| Grime et al, 2014 (59) | - | - | - | - | + | Umanodan et al, 2014 (100) | ? | - | + | - | + |
| Hasson et al, 2005 (49) | + | - | + | - | - | Wolever et al, 2012 (38) | ? | - | - | - | + |
| Heber et al, 2016 (71) | + | - | + | - | + | Yamagishi et al, 2008 (36) | ? | - | - | - | + |
| Hirsch et al, 2017 (101) | ? | - | - | - | + | Yuan, 2015 (103) | + | - | ? | - | ? |
Meta-analysis
The pre-post between-group effects for stress, depression, anxiety, burnout, insomnia, well-being, mindfulness, and alcohol consumption for all the studies including outliers are presented in Table 2. Below, the results of the eight meta-analyses are discussed according to outcome measure.
Table 2
Pre-post between-group effects for all outcomes including outliers. [CI=confidence interval.]
| Outcome measures | Comparisons (N) | Hedge’s g | 95% CI | Q-value | I2 (%) | Fail-Safe N |
|---|---|---|---|---|---|---|
| Stress | 22 | 0.54 | 0.35−0.72 | 145.89 a | 84.73 | 1635 |
| Depression | 17 | 0.30 | 0.18−0.42 | 40.10 a | 61.67 | 285 |
| Anxiety | 15 | 0.34 | 0.18−0.50 | 51.90 a | 71.57 | 259 |
| Burnout | 8 | 0.51 | 0.26−0.75 | 34.48 a | 79.16 | 231 |
| Insomnia | 7 | 0.70 | 0.25−1.15 | 48.14a | 91.88 | 116 |
| Well-being | 7 | 0.35 | 0.25−0.46 | 8.99 | 0.01 | 84 |
| Mindfulness | 5 | 0.42 | 0.24−0.60 | 4.03 | 0.00 | 40 |
Effects on stress. A pooled analysis of 22 RCT using random effects demonstrated a medium effect for reducing perceived stress, with g=0.54 (95% CI 0.35‒0.72, P<0.0001) (figure 2). However, the level of heterogeneity was high (I2=84.73%). The omission of two outliers (70, 71) decreased the effect to g=0.44 (95% CI 0.29‒0.59, P<0.0001) and slightly reduced the heterogeneity (I2=72.81%). No publication bias was identified.
Figure 2
Forest plot of the effect sizes of e-mental health interventions on stress, RE Model: random effects model, CI: confidence interval
Effects on depression. For the 17 studies with depression as an outcome, a small significant effect was observed (g=0.30, 95% CI 0.18‒0.42, P<0.0001) (figure 3). The level of heterogeneity was substantial (I2=61.67%), and one outlier was detected (72). After omitting this study from the analysis, the effect size dropped slightly to g=0.27 (95% CI 0.16‒0.38, P<0.0001), and the heterogeneity decreased to a moderate level (I²=50.35%). No publication biases were detected.
Effects on anxiety. A pooled analysis of 15 studies demonstrated a small effect on reducing anxiety (g=0.34; 95% CI 0.18‒0.50, P=0.0001) (figure 4). The level of heterogeneity was high (I²=71.57%), and one outlier was detected (71). After removing the outlier, the effect size dropped to g=0.28 (95% CI 0.14‒0.43, P<0.0001), but the heterogeneity remained in the moderate range (I²=58.11%). No publication bias was detected.
Effects on burnout A significant moderate effect was found for the eight studies aiming to relieve burnout (g=0.51, 95% CI 0.26‒0.75, P<0.0001) (figure 5). The level of heterogeneity was high (I²=79.16%), and no outliers were detected.
Effects on insomnia. Based on seven studies, a significant moderate effect was found (g=0.70, 95% CI 0.25‒1.15, P=0.0022) (figure 6). The level of heterogeneity was high (I²=91.88%), and one outlier was found (70). After removing the outlier, the effect dropped to g=0.52 (95% CI 0.39‒0.65, P<0.0001), and the heterogeneity decreased substantially (I²=0.01%). Meta-regression analysis did not identify any relationship of potential moderators and effect size with insomnia. No publication biases were identified.
Effects on mental well-being. The overall mean effect size for seven studies on well-being was g=0.35 (95% CI 0.25‒0.46, P<0.0001) (figure 7). The level of heterogeneity was low (I²=0.01%), and no outliers were detected. Using meta-regression analysis, we found no moderators for the pooled effect size. No publication biases were detected.
Effects on mindfulness. For mindfulness (five studies), a significant, moderate effect was found (g=0.42, 95% CI 0.24‒0.60, P<0.0001) (figure 8). The level of heterogeneity was low (I²=0.00%). The meta-regression analysis did not reveal any significant moderators of the effect size. No publication biases were found.
Effects on alcohol intake. The pooled analysis of two RCT with passive control groups demonstrated a small but nonsignificant effect on reducing alcohol intake, with g=0.13 (95% CI -0.23‒0.48, P=0.488). The level of heterogeneity was low (I2=0%); however, I2 is imprecise and has substantial bias when the number of studies is small (73).
Meta-regression analysis. Based on the results of the meta-regression analysis, we found that workplace recruitment had a significant negative association with the intervention effects in stress (slope=-0.47, 95% CI -0.80‒ -0.13, P=0.01), depression (slope=-0.31, 95% CI 0.01‒0.07, P=0.005), anxiety (slope=-0.33, 95% CI -0.63‒ -0.02, P=0.04), and burnout (slope=-0.56, 95% CI -1.03‒ -0.0, P=0.03). Age had a slightly positive influence on the effect size in e-mental health interventions for depression (slope=0.04, 95% CI 0.01‒0.07, P=0.005). An increased stress level among the participants demonstrated a positive significant association with the treatment effect (slope=0.55, 95% CI 0.23‒0.86, P=0.002). The presence of guidance moderated the treatment effect in stress-reducing interventions (slope=0.39, 95% CI 0.03‒0.75, P=0.04). Problem-solving therapy had a positive significant association with the treatment effect in cases of stress (slope=0.60, 95% CI 0.15‒1.04, P=0.01), depression (slope=0.35, 95% CI 0.13‒0.58, P=0.004), and anxiety (slope=0.40, 95% CI 0.12‒0.70, p=0.01). High study quality had a positive influence on the effect size in cases of depression (slope=0.34, 95% CI 0.15‒0.53, P=0.002), anxiety (slope=0.32, 95% CI 0.07‒0.62, P=0.046), and burnout (slope=0.61, 95% CI 0.39‒0.83, P=0.0005).
Subgroup analysis. Possible moderating effects can also be identified by explorative subgroup analyses (74) Exploratory subgroup analysis results are presented in Table 3. In the subgroups with open recruitment, significantly higher effects were found for stress, depression, anxiety, and burnout. In the subgroup participants >40 years, significantly higher effects were found for stress, depression, anxiety, and burnout. The presence of psychological symptoms significantly increased treatment effects in all mental health conditions. The presence of guidance and a higher quality of studies resulted in significantly higher treatment effects for interventions in all mental health areas (except well-being, where guidance was absent). The problem-solving therapy showed significantly higher treatment effects for stress, depression, anxiety, and burnout. For insomnia, significantly higher effects were achieved with mindfulness-based therapy.
Table 3
Subgroup analyses
| Outcome measures | Ncomp | Hedge’s g | 95% CI | I² (%) |
|---|---|---|---|---|
| Stress | ||||
| Recruitment type | ||||
| Workplace | 13 | 0.32a | 0.18-0.47 | 49.76 |
| Community | 9 | 0.79a | 0.47-1.12 | 89.13 |
| Age (years) | ||||
| >40 | 10 | 0.76a | 0.45-1.07 | 86.12 |
| <40 | 6 | 0.29b | 0.08-0.49 | 63.40 |
| Initial mental condition | ||||
| With | 9 | 0.84a | 0.55-1.13 | 87.49 |
| Without | 13 | 0.27a | 0.15-0.39 | 24.97 |
| Guidance | ||||
| With | 9 | 0.76a | 0.44-1.08 | 85.83 |
| Without | 13 | 0.38a | 0.19-0.56 | 74.54 |
| Therapy type | ||||
| CBT | 11 | 0.40b | 0.15-0.65 | 83.38 |
| Problem-Solving | 4 | 1.00a | 0.58-1.42 | 88.26 |
| Mindfulness | 3 | 0.64a | 0.36-0.92 | 0.00 |
| Positive Psychology | 1 | 0.41c | 0.02-0.80 | 0.00 |
| Others | 3 | 0.19c | 0.00-0.38 | 0.00 |
| Risk of bias | ||||
| High | 11 | 0.39a | 0.24-0.54 | 45.53 |
| Low | 11 | 0.67a | 0.36-0.98 | 89.91 |
| Depression | ||||
| Recruitment type | ||||
| Workplace | 8 | 0.11c | 0.007-0.22 | 0.00 |
| Community | 9 | 0.45a | 0.28-0.61 | 62.48 |
| Age (years) | ||||
| >40 | 10 | 0.47a | 0.33-0.61 | 35.48 |
| <40 | 5 | 0.13c | 0.02-0.24 | 0.00 |
| Initial mental condition | ||||
| With | 11 | 0.40a | 0.21-0.53 | 62.33 |
| Without | 6 | 0.20c | 0.03-0.35 | 41.42 |
| Guidance | ||||
| With | 6 | 0.48a | 0.33-0.63 | 0.00 |
| Without | 11 | 0.23b | 0.08-0.37 | 65.29 |
| Therapy type | ||||
| CBT | 9 | 0.18c | 0.04-0.33 | 18.40 |
| Problem-Solving | 5 | 0.54a | 0.37-0.70 | 41.15 |
| Positive Psychology | 1 | 0.19 | -0.00-0.39 | 0.00 |
| Others | 2 | 0.13 | -0.01-0.26 | 0.00 |
| Risk of bias | 9 | 0.15a | 0.061-0.24 | 0.00 |
| High | 7 | 0.52a | 0.32-0.71 | 59.96 |
| Low | ||||
| Anxiety | ||||
| Recruitment type | ||||
| Workplace | 9 | 0.18 | 0.04-0.32 | 31.33 |
| Community | 6 | 0.52a | 0.27-0.76 | 72.06 |
| Age (years) | ||||
| >40 | 9 | 0.41a | 0.20-0.62 | 69.71 |
| <40 | 4 | 0.16 | -0.06-0.40 | 52.19 |
| Initial mental condition | ||||
| With | 9 | 0.42a | 0.23-0.61 | 70.12 |
| Without | 6 | 0.20 | -0.04-0.44 | 59.13 |
| Guidance | ||||
| With | 5 | 0.48b | 0.16-0.80 | 76.29 |
| Without | 10 | 0.26 | 0.10-0.41 | 56.02 |
| Therapy type | ||||
| CBT | 8 | 0.18b | 0.06-0.30 | 0.00 |
| Problem Solving | 5 | 0.58a | 0.37-0.79 | 63.81 |
| Others | 2 | 0.32 | -0.50-1.15 | 79.72 |
| Risk of bias | ||||
| High | 9 | 0.18c | 0.04-0.31 | 30.41 |
| Low | 6 | 0.50a | 0.26-0.76 | 73.66 |
| Burnout | ||||
| Recruitment type | ||||
| Workplace | 2 | 0.07 | -0.17-0.32 | 0.00 |
| Open community | 6 | 0.64a | 0.40-0.87 | 71.17 |
| Age (years) | ||||
| >40 | 7 | 0.56a | 0.30-0.80 | 79.78 |
| <40 | 1 | 0.05 | -0.47-0.57 | - |
| Initial mental condition | ||||
| With | 6 | 0.60a | 0.30-0.88 | 80.88 |
| Without | 2 | 0.26c | 0.02-0.50 | 0.00 |
| Guidance | ||||
| With | 4 | 0.69a | 0.38-0.99 | 70.84 |
| Without | 4 | 0.33 | -0.01-0.68 | 79.08 |
| Therapy type | ||||
| CBT | 2 | 0.38b | 0.12-0.63 | 0.00 |
| Problem-Solving | 4 | 0.66a | 0.28-1.04 | 87.19 |
| Mindfulness | 2 | 0.26c | 0.02-0.50 | 0.00 |
| Risk of bias | ||||
| High | 4 | 0.23b | 0.07-0.38 | 0.00 |
| Low | 4 | 0.84a | 0.69-1.0 | 0.00 |
| Insomia | ||||
| Recruitment type | ||||
| Workplace | 1 | 0.70a | 0.42-0.97 | 0.00 |
| Community | 6 | 0.70c | 0.16-1.24 | 92.91 |
| Age (years) | ||||
| >40 | 5 | 0.80b | 0.21-1.40 | 94.14 |
| <40 | 2 | 0.47 | -0.14-1.07 | 63.38 |
| Initial mental condition | ||||
| With | 5 | 0.80b | 0.22-1.39 | 94.78 |
| Without | 2 | 0.45 | -0.16-1.06 | 58.5 |
| Guidance | ||||
| With | 3 | 1.00c | 0.00-1.99 | 96.55 |
| Without | 4 | 0.53 | 0.30-0.76 | 39.98 |
| Therapy type | ||||
| CBT | 3 | 0.95 | -0.18-2.08 | 94.86 |
| Problem-Solving | 3 | 0.45a | 0.29-0.61 | 0.00 |
| Mindfulness | 1 | 0.70a | 0.30-1.09 | 0.00 |
| Risk of bias | ||||
| High | 2 | 0.47 | -0.14-1.07 | 63.38 |
| Low | 5 | 0.80b | 0.21-1.40 | 94.14 |
| Well-being | ||||
| Recruitment type | ||||
| Workplace | 4 | 0.48b | 0.18-0.78 | 58.91 |
| Community | 3 | 0.30a | 0.16-0.44 | 0.00 |
| Age (years) | ||||
| >40 | 3 | 0.23c | 0.05-0.41 | 0.00 |
| <40 | 2 | 0.53c | 0.11-0.95 | 72.96 |
| Initial mental condition | ||||
| With | - | - | - | - |
| Without | 7 | 0.35a | 0.25-0.46 | 0.01 |
| Guidance | ||||
| With | - | - | - | - |
| Without | 7 | 0.35a | 0.25-0.46 | 0.01 |
| Therapy type | ||||
| Positive Psychology | 3 | 0.42b | 0.15-0.69 | 65.23 |
| Mindfulness | 2 | 0.46 | -0.03-0.96 | 63.02 |
| Others | 2 | 0.30 | 0.12-0.49 | 0.00 |
| Risk of bias | ||||
| High | 7 | 0.35a | 0.25-0.46 | 0.01 |
| Low | - | - | - | - |
| Mindfulness | ||||
| Recruitment type | ||||
| Workplace | 3 | 0.49a | 0.26-0.72 | 0.00 |
| Community | 2 | 0.61 | -0.32-1.54 | 64.00 |
| Age (years) | ||||
| >40 | 2 | 0.61 | -0.32-1.54 | 64.00 |
| <40 | 2 | 0.42c | 0.10-0.74 | 0.00 |
| Initial mental condition | ||||
| With | - | - | - | - |
| Without | 5 | 0.42a | 0.24-0.60 | 0.00 |
| Guidance | ||||
| With | 1 | 0.57a | 0.23-0.90 | 0.00 |
| Without | 4 | 0.36a | 0.16-0.57 | 0.01 |
| Therapy type | ||||
| Mindfulness | 4 | 0.43a | 0.20-0.66 | 17.14 |
| Positive Psychology | 1 | 0.44c | 0.05-0.83 | - |
| Risk of bias | ||||
| High | 2 | 0.33b | 0.10-0.55 | 0.00 |
| Low | 3 | 0.56a | 0.28-0.84 | 0.01 |
Discussion
To our knowledge, this systematic review with meta-analyses is the first that evaluated e-mental health interventions for every mental health condition in an occupational context. We found statistically significant moderate effects on stress, insomnia, and burnout and small treatment effects on depression, anxiety, well-being, and mindfulness compared with control groups with inactive control conditions. The meta-analyses of alcohol intake consisted of only two studies, and the effect on alcohol intake was small and nonsignificant.
From the 22 studies on stress included in the meta-analysis, it is evident that stress is the most widespread and growing concern for organizations and is the most common area of application of occupational e-mental health interventions. The significant moderate effect (g=0.54) on stress reduction is comparable to the findings of previous research conducted in the general population (16). The omission of two outliers (71, 70) decreased the effect (from g=0.54 to g=0.44), in accordance with the results of previous meta-analyses in the general population (14, 15). Community recruitment significantly enhanced the treatment effect (g=0.79), while workplace recruitment decreased the intervention effect. The positive treatment effect was significantly increased to a large effect size by the presence of guidance (g=0.76) and initial stress level (g=0.84). Problem-solving therapy also significantly increased the treatment effect (g=1.0).
For depression, we noted a small significant effect (g=0.30) when analyzing 17 studies. After omitting the outlier, the effect size slightly dropped (g=0.27), in accordance with the results of previous research in the general population (14, 16, 18, 19). As in the previous case, the effect size increased to a moderate size (g=0.52) in the subgroup with community recruitment. Higher participant age and better study quality slightly enhanced the effect sizes to g=0.45 and g=0.52, respectively.
In the case of anxiety, we analyzed 15 studies and found a small treatment effect (g=0.34), and the effect slightly dropped after the omission of the outlier (g=0.28). This result corresponds to the results of the meta-analysis in the general population conducted by Heber et al (14), while some previous systematic reviews reported large treatment effects on anxiety (18, 56) in the general population. However, it should be noted that anxiety was not a primary therapeutic target in all 15 interventions included in the meta-analyses on anxiety but was considered to be a secondary outcome in interventions on stress and/or depression. Similar moderating patterns were observed as in previous cases. The effect of the treatment was enhanced by community recruitment (g=0.52), problem-solving therapy (0.58), and the quality of the studies (g=0.50).
For burnout, after analyzing eight studies, we found a moderate effect size (g=0.51) with the same moderators as described earlier, namely, community recruitment (g=0.64) and quality of studies (g=0.84). Our findings are similar to those from the latest RCT on burnout in the general population, in which moderate-to-large effects for burnout were found (75). The meta-analyses of insomnia (seven studies), mental well-being (seven studies), and mindfulness (five studies) showed a moderate effect for insomnia (g=0.70), a small effect for mindfulness (0.42) and a small effect for mental well-being (g=0.35). The effectiveness research on reducing insomnia symptoms in the general population in a previous meta-analysis showed a moderate effect (24). Previous meta-analyses in the general population also demonstrated small effects for well-being and small effects for mindfulness (15, 16). The five studies on alcohol intake reported conflicting results. Two studies on alcohol intake reported small treatment effects (48, 76), while two other studies did not use comparable measures and reported improvements only for binge drinking and consumption reduction (34, 35). The fifth study (68) reported the opposite effect, namely, higher alcohol consumption in the intervention group. The pooled effect of the two studies on alcohol intake included in the meta-analyses was small and nonsignificant. Although current research findings regarding online interventions show improvements in drinking behavior in the general population (77), we recommend a meta-analysis that includes more comparisons to better estimate the effectiveness of e-health interventions on alcohol intake in the occupational context.
From the point of view of study design, we noted that open community recruitment enhances the treatment effect. This finding could be explained by the fact that participants who volunteer for community recruitment have a stronger intrinsic motivation and typically do not experience external expectations, eg, compliance with internal procedures in a workplace environment. Age moderated the effect on depression, and the subgroups with older participants showed significantly higher effects for stress, depression, and burnout. The enhancing role of older participants was previously shown (77) and could be explained by the higher self-disciplined engagement of participants that is required by most self-help interventions. However, this finding should be interpreted with caution, as RCT in younger participants in an occupational context are rare. Thus, we recommend that future research should include different age groups or directly address young employees to close the research gap. The initial level of psychological symptoms moderated the effect on stress, and analyses of subgroups showed the same tendency for all mental health conditions. Healthy populations usually have less room for improvement and, therefore, produce smaller treatment effect. This tendency was also previously reported in reviews on e-health interventions (12, 26).
The fact that guided e-health interventions yield better outcomes was already widely investigated in previous research (10, 18, 20). Our explanatory subgroup analyses also confirmed the advantage of the guidance for all mental health areas. However, the subgroup analyses were underpowered and should be interpreted accordingly.
Problem-solving therapy showed a significant moderating effect on effectiveness in cases of stress, depression, anxiety, and burnout; however, this result should be interpreted cautiously, as four of the six studies involving problem-solving therapy in the whole sample were performed by the same investigation team, which could cause a risk of author bias (71, 72, 78, 79). The subgroup analyses indicated that mindfulness and positive psychology also lead to higher effect sizes, in accordance with findings by other researchers (26). Our findings suggest the need to further investigate the effectiveness of these therapy approaches, as they appear superior to CBT according to our results. The education level of participants was predominantly high, so we could not assess its moderating effect due to homogeneity of the sample. Gender did not show any moderating effects on treatment effect.
Study quality had a moderating effect on depression, anxiety, and burnout. Subgroup analysis showed higher treatment effects in studies with a low bias risk in all mental health areas. A correlation between higher study quality and higher effect size could be explained with a larger range of those studies, careful design including systematic randomization approaches applied as well as low attrition rates that lowered the risk of bias in the quality assessment.
Our meta-analysis has several limitations. Although we identified 35 studies for the meta-analysis, we analyzed the effects per outcome, and some outcome groups consisted of only a few studies (eg, only 2 studies covered alcohol consumption). Some subgroup analyses covered a small number of studies, so it should be acknowledged that these analyses are underpowered, and the findings should therefore be interpreted with caution. While excluding outliers reduced heterogeneity among the studies, I2 remained large for some outcomes, implying a persistent heterogeneity among the studies that cannot be explained by this analysis. All studies were assessed as high risk due to the absence of blinding and self-reported outcomes; this absence is particularly important, as the absence of blinding may cause placebo effects in mental health studies. The majority of studies used a waitlist as a control condition, which may provoke nocebo effects. The status of waiting for a future treatment may weaken patients’ efforts to get better on their own initiative compared with natural conditions. Due to this fact, waitlist design studies can provide inaccurate estimates, resulting in stronger treatment effects (79). Approximately one-third of the articles for some outcomes (eg, insomnia and burnout) were written by the same investigation team, which may cause author bias. High dropout rates (>20%) are another common problem of e-health intervention studies, and this was reinforced by the present review; more than half of the studies (25) suffered from high attrition rates, while five studies did not report on attrition. Thus, it was impossible to appraise the bias associated with missing outcome data. Due to the heterogeneity of the studies, it was not possible to assess the actual amount of treatment and its role on effectiveness, as it was not reported consistently. Certain studies spanned weeks, some offered sessions or modules, and many did not report the extent of the treatment. Finally, the majority of the participants were highly educated at the university level, rendering the results less generalizable for other professional groups.
Based on the results discussed, we can conclude that e-mental health interventions can significantly improve mental health conditions in an occupational context with small to moderate treatment effects. Previous systematic reviews on employee population reported significant mental health improvements with small overall effects for stress, depression, and anxiety (26, 27). Higher moderate treatment effects were achieved for stress, insomnia, and burnout. These findings could potentially be explained by the lower degree of stigmatization linked to stress, sleep problems, or burnout than the stigmatization linked to mental health conditions that are closer to a clinical diagnosis of mental health disorder, such as depression, anxiety and alcoholism. This insight could be important for interventions conducted directly at the workplace. The hypothesis that interventions addressing less stigmatized mental health areas could lead to better treatment effects in an occupational setting should be examined in further studies. This insight could be important for interventions conducted directly at the workplace. Furthermore, the hypothesis that open community recruitment enhances the treatment effect should be tested. As an implication for practitioners, we recommend taking advantage of community recruitment settings, namely, by providing maximum voluntary data security and anonymity and time resources for the treatment for interventions conducted directly at the workplace. From a methodological point of view, we recommend testing the effectiveness of e-mental health interventions using objective outcomes such as biological markers so outcome assessment bias can be avoided. This approach would increase the evidence in this field. The effectiveness of e-mental health interventions should also be tested in less knowledge-based occupational sectors to determine whether occupational e-mental health interventions are effective for workers with lower education levels and for younger people and to examine how such interventions work for a more digital-savvy generation.
