The toll that mental illnesses take on individuals, families, companies, and economies, governments and non-governmental organizations around the world has been widely recognized. Mental health issues impact millions of people worldwide and are among the top causes of years lived with disability globally (1). They not only lead to disability but also contribute to unemployment and dependence on welfare benefits (2). Indeed, in 2010, the estimated yearly costs of mental illness were $2.5 trillion globally (3) and, in 2016, more than €450 billion for the European Union (4). It is expected that this number will rise to $6 trillion globally by 2030 (5), as indirect expenses related to mental disorders, such as sick leave days, have recently increased (6). As a result, promoting and addressing mental health in the workplace has gained strategic importance, with expected advantages for employees and companies.
Health promotion interventions in the workplace have been found to be successful in preventing mental health issues (7), with reviews also showing the effectiveness of in-person and online therapies in preventing and reducing mental health issues (8–11). However, these reviews also reported high heterogeneity between studies with respect to the approaches used for these interventions, their outcomes, and their methodological quality (10, 11). One of the challenges reported for digital mental interventions is the high rates of attrition, which is often under investigated.
Studies using return-to-work interventions, for example, showed high attrition rates (uneven attrition between intervention and control groups and significant loss to follow-up) and did not routinely assess compliance with the intervention (12). This was also seen in a systematic review of digital mental health interventions in the workplace (13), where many studies reported high levels of attrition, but where 11 of 32 studies failed to adequately describe any procedures for managing attrition or missing values, and where 11 (of 32) studies failed to provide power calculations for their sample sizes. A systematic review of universal and targeted workplace interventions for depression found specifically computerized interventions to have the highest mean attrition rate compared with other delivery methods (14). Low adherence and study attrition are regular and serious issues that could compromise the reliability of the results (15), therefore there is need to examine when and why these phenomena takes place.
The patient characteristics and trial factors that influence the overall uptake of an intervention can be learned by analyzing dropout rates and when they occur (16). Differential attrition, which is viewed as a serious threat to internal validity, occurs when attrition rates vary between treatment conditions (17). In a health behavior change trial, a slightly higher amount of attrition on average was found in the intervention conditions as compared to the control groups (18). Similar results were found for mHealth randomized control trials, where attrition in active conditions was, on average, roughly twice than that of controls (19). The authors speculated that the reason for higher attrition in treatment groups in mHealth might be an increased burden associated with their intervention. Another meta-analysis on smartphone-delivered mHealth interventions evaluated the different factors that affected attrition. It concluded that trials (i) delivering an acceptance-based intervention, (ii) providing participants with a financial reward, and (iii) reminding individuals to participate in the intervention had lower attrition rates compared to trials that used an online enrolment strategy (eg, by telephone instead of in-person enrolment) (15). Indeed, no baseline participant-level trait accurately predicted attrition. As far as we are aware, no meta-analysis has previously examined differential attrition in mental health interventions in the workplace and how this might vary across internet- and in-person-based interventions.
Therefore, the aim of this study was to systematically review and meta-analyze differential attrition of occupational mental health interventions, differentiating by delivery method (ie, internet- versus in-person based). The secondary research question of this meta-analysis was to assess which factors might be related to differential attrition in such studies.
Methods
This study was a systematic review with a meta-analysis, which followed the PRISMA statement (20). The study protocol was prospectively published in PROSPERO (CRD42022322394), and work was conducted under the EMPOWER (European Platform to Promote Well-being and Health in the Workplace) project, funded by the European Commission (21). The EMPOWER project researches the impact of an eMental health platform on preventing common mental health problems and reducing psychological distress in the workplace.
Search methods
On 1 February 2022, we conducted a search in the electronic databases of PubMed, Scopus and Web of Science Core. The searches were performed filtered by year range (2010–2022), type of study (randomized control trial, RCT), species (human), age (adult) and language (English, Spanish, Portuguese) when possible. To structure the eligibility criteria, the PICOS (Patient/Population; Intervention; Comparison, Outcome; Study design) approach was used. The combination of words from five different areas was used, namely mental health, intervention, workplace, implementation and study design. A full list of the search words can be found in the supplementary material (sjweh.fi/article/4173), appendix A. Additionally, experts from the EMPOWER Consortium were consulted for potential additional references during the months of January and February of 2023. An additional search was conducted in April 2024 following the same criteria.
Study population and article selection
The study population included adult employees who participated in a study focusing on preventing or reducing mental health problems in the workplace. We included those studies focusing on any kind of mental health problem except for addictions. Therefore, interventions were included if they aimed at promoting employee’s mental health in the workplace or reducing employee’s mental health symptoms. Additionally, studies were only included if they: (i) followed a RCT design, with control and intervention groups; (ii) included digital mental health interventions or traditional, non-digital mental health interventions or interventions containing both modalities that were implemented in a workplace environment for ≥6 weeks; (iii) reported response rates at baseline and/or attrition rates at post-treatment/follow-up or provided information that allowed us to calculate them; (iv) were published between 2010 and the search date and the publication language was either English, Spanish or Portuguese; (v) recruited participating employees from the workplace itself and the primary outcome was measured in employees.
Six reviewers collaborated on the selection of studies, hence two researchers independently reviewed 25% of the studies by title and abstract. After title and abstract screening, two researchers reviewed the manuscripts’ full-text for inclusion in the review. Researchers were blinded to each other’ decisions. A disagreement between individual judgments were resolved by a third independent party for the title and abstract screening and by a discussion between the reviewers for the full-text screening (see figure 1). Rayyan software was used throughout the study selection process. Prior to the screening, the six researchers evaluated a subset of eligible manuscripts to ensure compliance with inclusion and exclusion criteria.
Data extraction
Two researchers independently extracted the following data for selected studies via consensus methods: title and year of publication, aims and purpose, study population, country of study implementation, sample size, study methodology/methods, name of mental health intervention, main outcomes, intervention format, assessment and intervention times, setting/context, usage data of the intervention, attrition, number of participants contacted to participate in the study and number who agreed to participate, as well as attrition and non-participation reasons.
Quality assessment
The revised Cochrane risk of bias tool for randomized control trials (ROB 2) (22), was used to assess the quality of included trials. Two researchers (CM and RL) independently assessed each study according to screening questions with the five prescribed domains on risk of bias: arising from the randomization process; due to deviations from the intended interventions assignment to intervention); due to missing outcome data; in measurement of the outcome; and in selection of the reported result. Each screening question was assigned a low, medium, or high score, and then an overall risk of bias score was calculated based on these domains. Results are visually presented in supplementary table S1.
Statistical analysis
For each study, we created a 2×2 table with the number of participants who were lost to post-intervention and follow-up and the number who remained in the study in the intervention condition and control condition. After aggregating the available data, we determined the percentage of participants in each study’s intervention and control conditions who were lost to follow-up after randomization using the 2×2 tables. The risk ratio, where a value >1 indicates a higher attrition rate in the intervention condition and a value <1 indicates a higher attrition rate in the control condition, was the outcome of interest for the meta-analysis. In order to produce values that are symmetric around zero and whose sample distribution is better approximated by a normal distribution, the actual analysis was conducted using the log-transformed risk ratios. We then meta-analyzed the log-transformed relative attrition rates using a random-effects model with limited maximum-likelihood (REML) estimation for the degree of heterogeneity (23). Along with the outcomes of the Q-test for heterogeneity and the I2 statistic, we also present the (back-transformed) estimated average relative attrition rate and accompanying 95% confidence interval (CI). The funnel plot’s asymmetry was visually inspected for potential signs of publishing bias.
The link between the log-transformed relative attrition rates and various study parameters that may be associated to the degree of differential attrition was examined using meta-regression analysis with mixed-effects models using REML estimation (23). For baseline to post-intervention differential attrition, the following potential effect moderators were explored: time to post-intervention measure, length of the intervention, type of control group, compensation for participation, type of country according to the World Bank classification (low and middle income versus high income countries), type of intervention (in-person versus online), engagement restrictions, type of workers (white or blue collar), sample size at baseline, intervention (psychosocial, physical activity, occupational or multilevel), the specificity of the intervention (general mental health versus specific mental health concern), quality of the study, and if the inclusion criteria for the sample was suffering from mental health problems (ie, scoring above a certain threshold in a mental health diagnostic tool). For baseline to follow-up attrition, we calculated the potential effect of the same moderators, only modifying the time to post-intervention measure to the time to the last follow-up measure. The various variables were estimated univariately. All analyses were conducted in R (24) using the metafor package (25).
Results
Study characteristics
A total of 28 683 studies were included following the literature search and reference check (figure 1). After removal of duplicates, 21 670 were identified for abstract and title screening. Subsequently, 217 were included in the full-text screening from which 84 were included in the systematic review and 75 in the meta-analysis. The main reasons for exclusion in the full-text screening were different intervention goals, target group and study duration. Nine records were not included in the meta-analyses for the following reasons: one study did not report attrition rates (26), one study contained more participants at post-intervention than at baseline (27) and seven studies reported the same data in two papers (28–34).
A summary of the study characteristics can be found in supplementary table S2. The search resulted on 46 studies based on in-person delivered interventions, 29 studies of interventions delivered online, and 6 studies delivered through other formats. The studies encompass a wide range of target outcomes, including stress, depression, anxiety, burnout, and general mental health well-being. The mean participation rate was 42.51% [standard deviation (SD) 29.46%, range 1.27–100%], with a mean of 36.18% (SD 27.92%, range 1.27–92.5%) in online delivered interventions, 49.85% (SD 29.68%, range 2.08–100%) for in-person delivered interventions, and 13.69% (SD 1.94%, range 11.52–15.27%) for interventions delivered through other methodologies (eg, blended interventions or phone interventions). Most of the studies (N=42) used wait-list control group whereas 32 maintained the participants’ usual care as a control condition. Other control conditions were active control (6) and wait-list active control (2).
Qualitative analyses of the results
The reasons for attrition more often reported were lack of time (35–41), job situation change or work-conflict (37, 41–55) and personal health issues (45, 56, 57). In some studies, participants were excluded if they did not complete a certain percentage of the intervention [eg (40, 58–60),]. In the majority of the studies not all participants underwent the complete intervention [eg (42, 44, 61, 62),], and in many cases, some participants allocated in the intervention group did not participate in the intervention at all [eg (63–65),]. Additionally, many studies did not report usage data [eg (62, 66–68),].
Baseline to post-intervention differential attrition analyses
The average total attrition from baseline to post-intervention measure was 17.59% (SD 17.25, range 0–69.29%), with an average attrition of 19.77% (SD 20.08%, range 0–75.02%) for the intervention group and 15.68% (SD 16.25%, range 0–65.99%) for the control group. The random effects model showed a significantly higher attrition in the intervention compared to control group with a pooled risk ratio of 1.03 (see figure 2). In the case of online interventions, the average total attrition was 27.53% (SD 21.54%, range 0–75.02%), the average attrition in the intervention group was 31.74% (SD 25.62%, range 0–75.02%) and 24.51% (SD 20.63%, range 0–65.99%) in the control group. The random effects model online interventions also showed a higher attrition in the intervention group as compared to the control group with a pooled risk ratio of 1.04. In interventions delivered in person, the average total attrition was 12.93% (SD=11.99%, range=0–41.27%), the average attrition in the intervention group was 14.45% (SD=13.53%, range=0–47.49%) and in the control group was 11.50% (SD=11.56%, range=0–36.84%). The random effects model of in-person-delivered interventions did not show a higher attrition in the intervention compared to control group. Lastly, for interventions delivered through other methodologies, the average total attrition was 13.49% (SD 15.90%, range 0–40.45%), the average attrition in the intervention group was 13.37% (SD 17.25%, range 0–43.51%) and 13.30% (SD 14.41%, range 0–35.96%) in the control group. For other types of intervention, the random effects model did not result in a significant differential attrition. However, there was not a significant difference in differential attrition between interventions delivered online, in-person or through other methodologies. A summarized description of the main results can be found in supplementary table S3.
Figure 2. Forest plot for meta-analysis and subgroup meta-regression for baseline to post-intervention differential attrition.
Figure 3
Forest plot for meta-analysis and subgroup metaregression for baseline to last follow-up differential attrition.
Meta-regression univariate analyses showed no significant effects of any moderator variable (see supplementary tables S4 and S5). Mean attrition values from baseline to post-intervention for the different categorical moderators can also be found in supplementary table S6.
Baseline to follow-up differential attrition analyses
The average total attrition from baseline to follow up measure was 22.25% (SD 17.97%, range 0–67.78%), with an average attrition of 22.35% (SD 18.90%, range 0–69%) for the intervention group and 21.38% (SD 17.67%, range 0–66.25%) for the control group. The random effects model showed a higher attrition in the intervention compared to control group with a pooled risk ratio of 1.03. In the case of online interventions, the average total attrition was 32.12% (SD 19.43%, range 0–67.78%), the average attrition in the intervention group was 29.71% (SD 21.48%, range 0–69%) and 30.40% (SD 18.96%, range 0–66.25%) in the control group. The random effects model for online interventions did not show a higher attrition in the intervention compared to control group. In interventions delivered in person, the average total attrition was 20.21% (SD 16.60%, range 0–59.16%), the average attrition in the intervention group was 21.64% (SD 17.35%, range 0–64.41%) and 21.29% (SD 17.52%, range 0–64.41%) in the control group. The random effects model of in-person-delivered interventions showed a significantly higher attrition in the intervention compared to control group with a pooled risk ratio of 1.04. Lastly, for interventions delivered through other methodologies, the average total attrition was 14.54% (SD 17.46%, range 0–33.9%), the average attrition in the intervention group was 11.26% (SD 14.48%, range 0–27.59%) and 17.07% (SD 19.25%, range 0–37.93%) in the control group. For other types of intervention, the random effects model did not result in a significant differential attrition. However, there was not a significant difference in differential attrition between interventions delivered online, in-person and through other methodologies. A summarized description of the main results can be found in supplementary table S3.
Two of the tested moderators were found to significantly predict differential attrition from baseline to follow-up, namely compensation for participation and year of publication (see supplementary tables S4 and S5). Mean attrition values from baseline to follow-up for the different categorical moderators can also be found in supplementary table S6).
Discussion
In this meta-analysis, we examined differential attrition in occupational mental health interventions. Compared to the control group, participants in the intervention arm exhibited higher differential attrition from baseline to post-intervention and from baseline to the latest follow-up measurement. These results line up with previous meta-analysis on differential attrition in behavioral and clinical interventions (18, 19). The fact that participants in occupational mental health treatments are generally not blinded for the treatment allocation could be a speculative reason for the somewhat higher attrition rates in the intervention group. Participants may have consequently higher expectations for perceived treatment efficacy. If these expectations are not satisfied, participants in the intervention conditions may be less likely to complete follow-up assessments than participants in the control conditions who may have lower expectations to begin with. However, when attrition reasons were reported in studies, the reasons mentioned were rather lack of time (35–39), job situation change or work-conflict (37, 42–52) and personal health issues (45, 56, 57).
In the current meta-analysis, we did not observe a difference in the differential attrition rates between in-person and internet-based interventions. Indeed, when analyzed separately, differential attrition was only significant in internet-based interventions for the baseline to post-assessment measurement. However, differential attrition from baseline to follow-up assessment was found to only be significant for in-person interventions. Moreover, when compared, no significant difference was encountered between the differential attrition of both intervention types. Therefore, it seems that differential attrition affects occupational mental health intervention generally rather than the modality of intervention delivery.
When examining the potential moderators of differential attrition, we found that none of them were connected to the degree of differential attrition when looking at attrition from baseline to post-intervention. Nevertheless, for baseline to follow-up interventions, we could see a trend for a significant moderator effect for participants being compensated for their participation and year of publication. Variables such as time to follow-up or type of control condition were also examined in other meta-analyses and also failed to have a significant effect of the variables on differential attrition (18, 69). However, in a previous meta-analysis (15) the authors found an effect of financial compensation on differential attrition, aligning with our results. Even if the moderators evaluated in the meta-analysis already address many potential variations between trials (such as treatment length, length of evaluation period, type of intervention, compensation), there might be other additional factors that play a role in explaining differential attrition. Future research should expand the search for moderators to other potential variables that impact differential attrition. Identifying such variables could help tackle them and decrease differential attrition in studies and consequently increasing internal validity.
Another outcome of interest is the percentage of total attrition across studies. We encountered a mean attrition for baseline to post-intervention of 17.59% and of 22.25% from baseline to follow-up. When separating for in-person interventions, the attrition resulted 12.93% for baseline to post-intervention and 20.21% for baseline to follow-up. This is in line with previous literature on behavioral health interventions (18). For digital interventions, we found a mean attrition percentage of 27.53% from baseline to post-intervention and of 32.12% for baseline to follow-up. This outcome also aligns with previous findings in smartphone-delivered interventions for mental health problems (15), which show a mean percentage attrition of 24.1% for short-term follow-up and 35.5% at longer-term follow-up. These results would also be in line with a previous systematic review of universal and targeted workplace interventions for depression, which concluded that digital interventions had the highest mean attrition rate when compared with other delivery methods (14). Moreover, these values can serve as a reference for future studies.
This systematic review and meta-analysis has some limitations that must be taken into account. First, our subgroup analyses on study-related factors were based on studies that used distinct samples with varying occupations and different interventions, targeting different mental health issues. Therefore, it is not obvious whether these findings may be applied to certain target mental health problems or particular interventions and occupations. While we added moderator analyses controlling for the effect of the intervention delivered or whether the goal of the intervention was to improve mental health in general or a specific mental health condition, our analyses may not have captured the effect of the heterogeneity of the studies included. Additionally, the results of this study are limited to occupational mental health interventions published from 2010 onwards and in certain languages, which may lead to publishing and cultural biases in our results.
To conclude, our research suggests that differential attrition rates are frequent and might be problematic in RCTs for occupational mental health interventions. The findings have a number of consequences for scientists, decision-makers, and clinicians. First, it is reasonable to assume that between one-quarter and one-third of participants in future trials of smartphone therapies might drop out from the study. When creating recruiting plans and doing a priori power analysis, this amount of lost data should be taken into account to ensure that the RCT’s statistical validity is not jeopardized. Moreover, statistical analyses should be adapted accordingly. When differential attrition takes place, a usual complete cases analysis would lead to biased results, whereas using an expectation-maximization algorithm could tackle this bias if the attrition mechanism is accessible (70). Although our results were unable to shed light on the type of participants most likely to drop-out from the RCT, there may be some strategies that researchers might apply to potentially reduce attrition (eg, use personalized enrollment methods, gamification of the intervention). Moreover, as we found that compensating for the participation may influence attrition, this could potentially be a strategy to increase study adherence in occupational mental health intervention studies. For well-being and mental health interventions through mobile apps, gamification can be a useful and effective platform. It may also increase motivation and decrease attrition (71).
