Mental illness constitutes a substantial burden of disease to a large part through its major impact on years lived with disability (1). Relative to many other disabling illnesses, it disproportionally affects individuals of working age and is the major cause of social security disability benefits and work disability in the US as well as Europe (2–4). Individual placement and support (IPS) is a model of vocational rehabilitation that aims to help people with severe mental illness obtain and maintain competitive employment in ordinary jobs (5). IPS follows a strict manual to secure integration of supported employment with psychological treatment. The following eight principles are incorporated in the approach to vocational rehabilitation: (i) eligibility based on the patient’s own choice, (ii) focus on competitive employment, (iii) integration between mental health and employment services, (iv) job support guided by the patient’s preferences, (v) financial counseling about social security benefits, (vi) rapid job search initiation, (vii) employment specialists’ engagement in systematic job development, and (viii) individualized and continuous job support (6). The model has demonstrated successful results in 23 randomized controlled trials (RCT) worldwide (7, 8), and in the US and elsewhere, governments have taken steps to encourage the expansion of IPS (8, 9).
In the current study, we present the results of a large multicenter RCT from Norway, a policy context with high job security, low unemployment, and a comprehensive welfare system. The trial was commissioned as a collaboration between the Directorate of Health and the Directorate of Labor and Welfare as part of a national initiative to implement IPS on a large scale.
The aim of the study was to evaluate the effect of IPS compared to high-quality treatment as usual offered to people with moderate-to-severe mental illness. The main outcome was competitive employment rate, derived from objective registry data with no loss to follow-up. Secondary outcomes were levels of self-reported psychological distress, somatic symptoms, functioning, well-being and quality of life. The study further involved multiple sites from both rural and urban areas, had a high number of participants, and long-term follow-up for the main outcome.
Methods
Design and setting
The trial was a multicenter RCT comparing IPS to high-quality treatment as usual offered to people with moderate-to-severe mental illness. Participants were recruited to any of the six IPS centers from regional primary and secondary mental healthcare settings while they were undergoing treatment for moderate-to-severe mental illness. To be eligible for participation, patients were required to have at least one diagnosed psychiatric disorder (10). Participants also had to be currently out of the labor market but with an expressed desire to work. The only exclusion criterion was insufficient Norwegian language skills to answer the questionnaires. Inclusion started on 1 October 2013 and ended on 31 October 2014. The detailed study protocol is published and available online (11).
Interventions
Participants were randomized to one of two interventions: (i) IPS, following the structured and manualized approach focused on competitive employment, or (ii) a high-quality version of treatment as usual, which involved a prioritized spot in a vocational rehabilitation scheme, primarily work with assistance and/or a traineeship in a sheltered business. While IPS follows a “place-and-train” approach to vocational rehabilitation, the intervention schemes in the control condition were more in line with a “train-and-place” approach to vocational rehabilitation (11). These interventions, although they also aim to promote competitive employment, are based on stepwise approaches including prevocational training and non-competitive work in the process. In work with assistance, clients are assigned a personal facilitator who provides support to find a suitable job using various subsidized schemes and employer incentives on the way to competitive employment. Traineeship in a sheltered business involves an even stronger focus on “train-and-place” and includes work training in a sheltered environment, where the work capability of the clients is tested and tasks are adapted to the individual’s skills and challenges.
Participants
In total, 410 participants entered the trial and were randomized, 56% (N=229) to the intervention group and 44% (N=181) to the control group (figure 1). The unbalanced allocation (a 2:1 randomization ratio the first five months of recruitment) ensured that the IPS centers could run according to maximal capacity.
Ethical considerations
The project was submitted to the Norwegian Regional Ethical Committee (REC) on 28 May 2013 (project no. 2013/960), which found the project did not fall under the Health Research Act (12) and was outside the remit of the REC as the main outcome (competitive employment) was not directly health-related. The project was thereby exempted from evaluation and referred to the Norwegian Social Science Data Services, where the required permissions were acquired on 4 October 2013 (project no. 34989). The research was carried out in compliance with the Helsinki Declaration. Personal confidentiality was guaranteed, and informed consent was signed by each participant with emphasis on the right to withdraw from the study at any time without any explanation.
Recruitment and randomization
The randomization procedure strictly adhered to the formal requirements of adequate randomization at all times (11). Eligible and willing participants signed the informed consent and completed the baseline questionnaire. After randomization, participants were informed about group allocation. Participants in the intervention group were given a date for their first session, and participants in the control group were referred to the Norwegian Labor and Welfare Administration (NAV) for a prioritized spot in a vocational rehabilitation scheme.
Outcome measures
The primary outcome of the study was competitive employment rate at 12-months follow-up. Using data from NAV’s State Register of Employers and Employees (SREE), we could determine if a person was engaged in competitive employment, operationalized as being registered in the SREE, with no loss to follow-up. The registry gives an overview of individual employment and employers. Employers are required to report job spells to the SREE, including the day of entry and exit for each job. NAV records the SREE information continuously, and data on intervention participants was made available to us for every month up until 18 months. The registry provides a conservative, yet reliable and complete measure of registered ordinary employment for all participants at any time and is more objective and accurate than self-reported employment or logbooks used to define the outcomes in most previous IPS-studies. In addition, we collected self-reported information on competitive employment through questionnaires at 6- and 12-months follow-up to compare with previous IPS-studies.
Questionnaire-based secondary outcome measures were levels of psychological distress, using the Hospital Anxiety and Depression Scale (HADS) (13, 14); level of functioning, using the World Health Organization Disability Assessment Schedule (WHODAS) 2.0 which is a generic measure of functioning based on the ICF framework (15); frequency and severity of subjective health complaints, using the Subjective Health Complaints Inventory (16); health-related quality of life (QoL), using the Euro-Qol Visual Analog Scale which is a vertical scale ranging from 0 (“worst imaginable health state”) to 100 (“best imaginable health state”) (17); and global well-being, using a Cantril ladder scale (18) ranging from 1 (the worst life possible) to 10 (the best life possible). Self-report questionnaires were administered to the patients at baseline and after 6 and 12 months.
The Mini-International Neuropsychiatric Interview (MINI) 6.0.0 was administered to screen for psychiatric diagnoses according to criteria of the DSM-IV and ICD-10 (19). The MINI 6.0.0. is a valid and reliable screening instrument intended for research purposes (19, 20).
Fidelity measures
The IPS-25 Fidelity Scale (21) was used to assess the quality of implementation. The scale measures adherence to the IPS principles and has shown good predictive and discriminative validity in previous studies in the US context (21). The scale has also been used in Sweden (22), and the older version of the scale was used in a European six-center trial supporting the predictive validity in a European context (23). The psychometrics of the scale is yet to be formally tested in a Norwegian context but was used to capture fidelity and ensure comparability with previous international IPS trials. Teams of trained evaluators follow detailed instructions, and the ratings are done based on interviews, team meeting observations, and document reviews. Total score ranges from 25–125, and critical cutoff point for being recognized as IPS is >74 (21). Most centers were assessed at three time points during the project period (2014–2016). At the first measurement, four of the nine teams (spread across six centers) were just below cutoff, but on the second and third assessment, all teams scored fair (74–99 points), good (100–114 points), or exemplary (115–125 points) on fidelity. Eight of the nine teams improved their fidelity scores steadily throughout the project period. The fidelity results will be described in further detail along with other process measures in a subsequent process evaluation paper.
Sample size
Our sample size calculation was based on input-data from previous IPS studies. In non-US studies of IPS, the mean competitive employment rate has been found to be 50% for IPS and 20% for controls (24), with studies in the lower range showing employment rates of 22% for IPS and 11% for controls (25). To identify a group difference given the first mean scenario, 39 participants in each group would be needed. Given differences in the lower range, 178 participants in each group would be needed (with a 5% significance level and power of 80%). We assumed that our study would include a more heterogeneous study population than previous IPS studies, and also that the control condition on average would have higher quality of care. Both assumptions should reduce effect differentials, and we therefore planned for stronger statistical power and aimed to recruit 400–500 participants [for details, see the protocol paper (11)].
Statistical analysis
For the main effect analysis, we observed crude employment rates of participants in the two groups. In separate multiple linear regression analyses, we also examined effects of the intervention including adjustment for group differences in age, gender, calendar year and month, and site indicators. This alternative approach did not yield any noteworthy changes in results. The effects of IPS on secondary outcomes were examined using multiple mixed-effects regression analyses with maximum likelihood estimation. Estimated marginal means, with 95% confidence intervals (CI), were based on the mixed effects regression models. Self-reported employment was adjusted for non-response using inverse probability weights (IPW). The IPW were calculated based on logistic regression models adjusting for variables predicting non-response, with a P-value of <0.10. Analyses adhered to the “intention-to-treat” principle. To test for potential subgroup effects of IPS, an analysis of the prespecified subgroups “moderate versus severe mental illness” was conducted (11). The author in charge of the data analysis was blinded for intervention assignment. Data were analyzed using the statistical package StataIC (Stata Corp, College Station, TX, USA) version 14 and 15.
Results
Figure 2 outlines the main outcome measure, competitive employment rate, for each month. Data on this outcome measure were complete for all participants up to 18 months post-inclusion. However, for secondary outcomes based on self-report, 47% of the participants at 6-months follow-up, and 44% of the participants at 12-months follow-up returned their questionnaires (see figure 2).
Figure 2
Average employment rates in both groups. Development over time from 44 months prior to inclusion up to 18 months after inclusion.
Participant characteristics
Psychiatric screening
MINI 6.0.0 was administered at baseline and showed that depression as well as anxiety and psychotic disorders were the most common diagnoses (table 1). In total, 45% (N=136) of the participants fulfilled the criteria for what previous IPS trials have defined as severe mental illness (ie, psychotic or bipolar disorder with or without comorbid substance abuse/dependency), while 55% (N=164) were categorized as moderate mental illness (primarily affective disorders).
Table 1
Demographic and clinical characteristics at baseline. [SD=standard deviation; IPS=individual placement & support]
| Variable | Control | IPS | P-value a | ||||||
|---|---|---|---|---|---|---|---|---|---|
|
|
|
||||||||
| N | % | Mean | SD | N | % | Mean | SD | ||
| Age at inclusion | 34.92 | 10.78 | 35.11 | 10.77 | 0.858 | ||||
| Years of illness | 11.49 | 9.55 | 10.42 | 8.90 | 0.302 | ||||
| Gender (female) | 85 | 47.0 | 114 | 49.8 | 0.570 | ||||
| Marital status (married/cohabiting) | 45 | 25.3 | 43 | 19.2 | 0.143 | ||||
| Country of birth (Norway) | 157 | 86.7 | 196 | 85.6 | 0.738 | ||||
| Higher education (yes) b | 40 | 23.5 | 55 | 25.0 | 0.738 | ||||
| Reading- and writing difficulties | 33 | 18.4 | 37 | 16.4 | 0.585 | ||||
| Previously involuntarily committed | 43 | 24.2 | 60 | 27.6 | 0.432 | ||||
| Psychiatric diagnoses c | |||||||||
| Psychotic | 46 | 41.4 | 65 | 58.6 | 0.732 | ||||
| Bipolar | 15 | 26.3 | 42 | 73.7 | 0.006 | ||||
| Major depression | 67 | 40.9 | 97 | 59.1 | 0.564 | ||||
| Anxiety | 65 | 39.2 | 101 | 60.8 | 0.232 | ||||
| Alcohol/drug abuse | 36 | 48.0 | 39 | 52.0 | 0.279 | ||||
| Other | 11 | 32.4 | 23 | 67.6 | 0.202 | ||||
Primary outcome measure
Registry data from the SREE provided objective and direct indicators of work participation: start and stop dates for participants’ work engagements, retrieved monthly during the full time-frame of the trial. Information on benefits, such as work assessment allowances (WAA), disability benefits or unemployment benefits, were added to gain more detailed information on the participants’ degree of work participation.
There could be incomplete registrations of both new and terminated jobs in the SREE registry, but this measurement error should be non-differential for the intervention and control group. With that assumption, the measurement error should not bias estimates, but make it harder to demonstrate any true effect of the intervention. In certain cases, jobs appeared to have ceased without a registered stop-date. As table 2 shows, about 19% of the participants were registered as employed at baseline, and, as expected, the proportion is very similar in the treatment and control groups. Many of these 76 individuals had a weak attachment to the labor market at inclusion, eg, 11 were sick-listed, 32 combined job and WAA, and 8 were on graded disability benefits.
Table 2
Competitive employment and benefit recipiency at inclusion, 12-, and 18-month follow-up. [IPS=individual placement & support.]
| Baseline (at inclusion) | 12-month follow-up | 18-month follow-up | |||||||
|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|||||||
| Control (N=181) | IPS (N=227) | P-value | Control (N=181) | IPS (N=227) | P-value | Control (N=181) | IPS (N=227) | P-value | |
|
|
|
|
|||||||
| Mean a | Mean a | Mean a | Mean a | Mean a | Mean a | ||||
| Employment | 0.188 | 0.185 | 0.942 | 0.271 | 0.366 | 0.042 | 0.271 | 0.374 | 0.027 |
| Sick-leave | 0.072 | 0.048 | 0.320 | 0.022 | 0.044 | 0.227 | 0.022 | 0.070 | 0.025 |
| Disability pension | 0.122 | 0.119 | 0.936 | 0.144 | 0.145 | 0.961 | 0.160 | 0.159 | 0.964 |
| Work assessment allowance (WAA) | 0.547 | 0.502 | 0.370 | 0.613 | 0.564 | 0.316 | 0.569 | 0.529 | 0.416 |
| WAA and employment | 0.099 | 0.062 | 0.159 | 0.127 | 0.172 | 0.212 | 0.110 | 0.167 | 0.102 |
| Unemployment benefits | 0.022 | 0.053 | 0.112 | 0.033 | 0.031 | 0.895 | 0.028 | 0.035 | 0.664 |
| Social security benefits | 0.166 | 0.163 | 0.941 | 0.099 | 0.101 | 0.950 | 0.102 | 0.090 | 0.732 |
Table 2 shows an increase in participants receiving disability benefits in both the control and the intervention group over time. This tendency was also observed for recipiency of WAA from inclusion up to 12 months. There was however also an increase in the proportion of participants who were working. These two categories, work participation and receiving WAA, are not mutually exclusive. For instance, a person may be able to work 50% and receive disability pension or WAA for the remaining 50%. At 18-months follow-up, there was a larger share of sick-listed participants in the IPS than the control group. This should be expected given that employment (which is a requirement for sick-listing) was more common in the IPS group at 18 months. [If we consider the subsample of employed individuals at 18 months, sickness absence is more frequent in the IPS than the control group, but the difference is not statistically significant.] Results show a large increase in the proportion of participants in competitive employment in the IPS group (figure 2). While the proportion of employed participants was similar in the two groups at inclusion, there was a difference in favor of the intervention group of 9.5 percentage points (proportions are 36.6 versus 27.1) at 12-months follow-up.
Table 3 presents the results from the subgroup analysis of diagnostic groups (moderate versus severe mental illness). While the proportion of employed participants was slightly higher in the group with moderate mental illness at baseline, both groups showed a similar effect of IPS on competitive employment at 12- and 18-months follow-up. The tests of comparison within each subgroup did not reach statistical significance, probably due to an insufficient number of participants in the two subgroups.
Table 3
Subgroup analysis of participants with severe mental illness (psychotic or bipolar disorders) and moderate mental illness (primarily affective disorders). Competitive employment at inclusion, 12-, and 18-month follow-up. [IPS=individual placement & support.]
| Employment | Baseline (at inclusion) | 12-month follow-up | 18-month follow-up | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|||||||||||||
| Control | IPS | P-value | Control | IPS | P-value | Control | IPS | P-value | |||||||
|
|
|
|
|||||||||||||
| N | Mean a | N | Mean a | N | Mean a | N | Mean a | N | Mean a | N | Mean a | ||||
| Severe mental illness | 51 | 0.157 | 85 | 0.118 | 0.517 | 51 | 0.235 | 85 | 0.353 | 0.153 | 51 | 0.255 | 85 | 0.388 | 0.113 |
| Moderate mental illness | 76 | 0.211 | 88 | 0.193 | 0.784 | 76 | 0.276 | 88 | 0.375 | 0.182 | 76 | 0.263 | 88 | 0.375 | 0.128 |
Self-reported employment
In the unadjusted analyses of self-reported employment, a significantly higher proportion in the IPS group (43%) reported to have been in competitive employment compared to the control group (27%) during the past 12 months (P=0.013). When adjusted for non-response using IPW, the self-reported employment rates changed to 41% and 28% for the IPS and control group respectively. The difference was still statistically significant (P=0.038).
Secondary outcome measures
The IPS group reported significant improvements in psychological distress, symptoms of depression, subjective health complaints, functioning, health-related QoL, and global well-being, compared to the control group at 12-months follow-up (table 4).
Table 4a
Health and well-being. Mean scores at baseline and 6-month follow-up. Adjusted for baseline and missing data. [CI=confidence interval; HADS=Hospital Anxiety and Depression Scale; SHC=subjective health complaints; WHODAS=World Health Organization Disability Assessment Schedule; QoL=quality of life.]
Table 4b
Health and well-being. Mean scores at 12-month follow-up. Adjusted for baseline and missing data. [CI=confidence interval; HADS=Hospital Anxiety and Depression Scale; SHC=subjective health complaints; WHODAS=World Health Organization Disability Assessment Schedule; QoL=quality of life.]
Missing data
Missing data on the secondary outcomes were adjusted in mixed effects regression models, using maximum likelihood estimation (MLE). Possible systematic differences between those who responded to questionnaires at 6- and 12-months follow-up and those who did not, were investigated by comparing baseline characteristics. Results showed that non-responders reported better function, less subjective health problems and fatigue, better health-related QoL and more problems with substance abuse at baseline. There were no indications of a differentiated pattern of drop-out between the IPS and control groups. Due to practical difficulties, the psychiatric interviews (MINI) were not administered to all participants. However, we have no reason to believe that there were any systematic patterns in the missing interviews.
Discussion
The primary aim of this trial was to investigate whether IPS resulted in a higher rate of competitive employment compared to high quality usual care among people with moderate-to-severe mental illness in a societal context characterized by high job security, low unemployment, and a comprehensive welfare system. At 12-months follow-up, registry data revealed that 36.6% of participants in the IPS group and 27.1% of participants in the control group were in competitive employment, while the difference was slightly higher (37.4% versus 27.1%) at 18-months follow-up. Furthermore, a subgroup analysis showed that the employment rates were similar in participants with moderate and severe mental illness, which indicates that IPS has an effect beyond the original target group of patients with severe mental illness.
The secondary aims of the trial involved investigating potential effects of IPS on psychological distress, symptoms of depression, and improvements in health complaints, functioning, health-related QoL, and general well-being. IPS yielded positive effects on all the secondary outcomes compared to the control condition.
Study context
The present results need to be interpreted with regard to study context. The Norwegian welfare regime is more extensive and also provides higher financial reimbursements than most welfare systems (26). Moreover, an important characteristic of the Norwegian labor market is relatively strong employment protection legislation (27) leading to high job security. There is reason to believe that a higher level of job security is a challenge for employment specialists regarding employers’ willingness to hire. Norway is a small, open economy, where many firms are exposed to international competition. As in other Nordic welfare states, the unemployment rate is low but the wage structure is rather compressed, which considerably limits the demand for low productivity labor. Of all OECD countries, Norway has the highest sickness absence rates and financial expenses due to absence from working life (26). Such contextual factors might reduce the potential for effects of interventions to increase work participation and could thereby strengthen the salience of the observed effect of IPS on work participation in the present study.
Comparison with international IPS trials
The effect of IPS on work participation among people with severe mental illness has a large and growing evidence base in 23 RCT studies worldwide so far (8). The majority of these were conducted in various US states, but several have emerged from European and Eastern countries in the last decade. IPS trials with high model adherence have in a meta-analysis demonstrated employment rates of 61% as compared to 23% in groups receiving traditional rehabilitation programs (7). These rates are primarily based on IPS trials in the USA, which on average show greater effect on employment than studies from other western countries (24). In a large European trial involving six different countries, 55% of IPS participants and 28% of control group participants receiving traditional work rehabilitation programs gained employment (28). In four of these six countries (England, Italy, Switzerland and Bulgaria) IPS was more effective than the traditional work rehabilitation programs offered. In the other two countries, Germany and the Netherlands, no statistically significant differences were found between IPS and control groups. The authors suggested work characteristics, policies and the welfare regimes in these two countries explained the lack of an effect. Only one RCT has previously evaluated the effect of IPS for people with severe mental illness in Scandinavia. The Swedish study (29) showed a significant effect of IPS on employment at 18-month follow-up, with 46% of the IPS-group participants employed compared to 11% in the control group.
Despite the statistically significant and robust results from the current trial, our rates of competitive employment are lower than in most previous IPS trials. One possible explanation could relate to our somewhat different inclusion criteria compared to those used in previous trials. We included a broader group of participants by allowing patients with any psychiatric disorder to be included, and not only patients with a psychotic or bipolar disorder like in most previous IPS trials. Still, the subgroup analysis demonstrated similar employment rates in the two subgroups, indicating that the inclusion criteria did not influence employment rates substantially.
Another, more credible, explanation for the somewhat lower employment rate is our primary outcome data and definition. All previous IPS trials have assessed employment outcomes using self-report from participants and/or logbooks, with threshold values varying from one hour to one day of employment throughout the whole follow-up period. In contrast, by using registry data from the SREE, employments not expected to exceed NOK1000 (approximately EUR105) per year per employer will not be subject to notification. Minor jobs below this threshold will therefore not be registered. Further exceptions apply to freelance and self-employment. This implies a comparatively stricter definition of work than in previous trials based on self-report. A main advantage of registry data is its objectivity. Registry data also secures information from all participants over the entire duration of the trial. Compared to the relatively high drop-out rates when relying on self-report, this removes an important source of attrition bias.
In order to compare results more directly with previous international studies, we also included self-reported employment. Corresponding with the results from the registry data, participants receiving IPS reported considerably higher employment rates compared to the control group, but rates using self-reported data were higher.
The control group in the current study was found to have a higher competitive employment rate than most international IPS trials. This could be partly due to the quality of services provided to control group participants. Not only were services provided to the control group a high-quality version of usual care, which included prioritized access to traditional employment services, but there may also have been a “spillover-effect” from IPS into the services providing vocational rehabilitation to the control group. This trial was conducted in 6 of 19 Norwegian counties, and was among the largest IPS trials worldwide. Moreover, the awareness of IPS and its impact on work participation found in international studies increased among service providers during the timeframe of this trial. It cannot be ruled out that those providing work rehabilitation services to the control group may have incorporated elements from IPS in their services and that this has had an impact on their results.
Strengths and limitations
The main strength of the study is the multicenter RCT design and the large sample size, providing robust, complete and objective outcome data on the effect of the intervention as deployed in an ecologically valid setting. The research group was not involved in running the sites or the patient treatment, which ensured an independent evaluation. All the IPS centers scored consistently high on fidelity, yet the scores differed somewhat between centers. The fidelity assessments revealed an increase in fidelity throughout the trial. Alongside the trial, an extensive process evaluation was carried out to investigate implementation quality, barriers and facilitators, contextual factors and the participants’ experiences. These findings will be published in a subsequent paper. Limitations of the study include inaccurate information on hours worked and the relatively high loss to follow-up on self-reported outcomes. The latter was, however, mitigated by applying maximum likelihood estimation (MLE) and IPW in the analyses of self-reported outcomes.
Implications
In line with recent OECD recommendations (26), the IPS program integrates mental healthcare and rehabilitation services and incorporates a focus on both work and health. The IPS trial was uniquely co-commissioned by the Norwegian health and welfare authorities as part of a national initiative to implement IPS on a large scale. The experience of successful collaboration between academic researchers and policymakers, where a novel policy scheme and intervention underwent rigorous scientific evaluations before large-scale implementation, has important policy implications and should inspire further use of trials in this sector. Beyond strengthening the evidence base, such collaborations can help bridge the gap between knowledge and practical application (30).
