Original article

Scand J Work Environ Health 2015;41(6):529-541    pdf

doi:10.5271/sjweh.3528

Implementation of the Danish return-to-work program: process evaluation of a trial in 21 Danish municipalities

by Aust B, Nielsen MBD, Grundtvig G, Buchardt HL, Ferm L, Andersen I, Lund TL, Jelle MOC, Andersen MF, Hansen JV, Tverborgvik T, Helverskov T, Bjorner JB, Rugulies R, Ørbæk P, Winzor G, Bültmann U, Poulsen OM

Objectives The aim of this study was to evaluate the implementation of the Danish national return-to-work (RTW) program in 21 Danish municipalities.

Methods We conducted a structured process evaluation on (i) reach and recruitment, (ii) fidelity, (iii) dose-delivered, (iv) dose-received, and (v) context by formulating 29 implementation criteria and analyzing qualitative and quantitative data from administrative records, interviews, field notes, and questionnaires.

Results All municipalities integrated the basic features of the RTW program into the existing framework of the sickness benefit management system to an acceptable degree, ie, establishment of RTW teams, participation of RTW team members in the training courses, and following the general procedures of the program. However, the level of implementation varied considerably between the municipalities, particularly with respect to fidelity (defined as implementation consistent with the principles of the interdisciplinary RTW process). Five municipalities had high and eight had low fidelity scores. Similar large differences were found with regard to dose-delivered, particularly in the quality of cooperation with beneficiaries, employers, and general practitioners. Only 50% of the first consultations with the RTW coordinator were conducted in time. Among participants who were employed when their sickness absence period started, only 9% had at least one meeting with their workplace.

Conclusion It was feasible to implement the basic features of the Danish RTW program, however, large variations existed between municipalities. Establishment of well-functioning interdisciplinary RTW teams might require more time and resources, while ensuring early assessment and more frequent cooperation with employers might need more general adjustments in the Danish sickness benefit system.

This article refers to the following texts of the Journal: 2014;40(1):47-56  2012;38(2):120-133

Returning to work after long-term sickness absence is a multifaceted and dynamic process involving various stakeholders, such as the sick-listed person, the employer, the insurance agency, and the healthcare system (1, 2). Interventions to improve return to work are often complex (3) as they contain a number of interactive components that need to be coordinated. The evaluation of complex interventions requires both a theoretical understanding of how the intervention is supposed to cause change and a thorough process evaluation to identify implementation problems so that weak links in the causal chain can be identified and strengthened (35).

In 2008 the Danish government launched an action plan to reduce sickness absence and improve labor market participation, encompassing 39 initiatives including the Danish return-to-work (RTW) program. The Danish National Research Centre for the Working Environment (NRCWE) was appointed to develop the program in accordance with the action plan and previous experiences from national and international intervention studies. The aim was to examine whether it is possible to implement a RTW program within the existing sickness benefit framework in Danish municipalities and whether the program reduces sickness absence and promotes labor market participation (6).

The program contained a stratified cluster controlled trial in 21 municipalities, with a nested randomized controlled trial (RCT) in 3 of these 21 municipalities. In two previous articles about the effect of the intervention in the 3 RCT municipalities, we found large differences between municipalities with regard to effect on the length of sickness absence (7) and time-to-self-support (8). In this article, we examine whether and to what extent it was possible to implement the Danish RTW program in all 21 participating municipalities.

Methods

Legislative context

The Danish sickness benefit scheme covers wage earners, self-employed and unemployed Danish residents. The scheme has undergone many changes (9) and continues to do so; the most recent changes came into force in 2014. In the following, we describe the relevant regulations during the time of the study: Employers pay the full wage or partial compensation for a certain period in the beginning of the sickness absence spell. This period, the so-called “employer period”, was 21 days when the study started in 2010 and was extended to 30 days in January 2012, which was within the timeframe of the RTW program. After the employer period, employers can claim compensation for a part of the wage from the local municipality.

The municipal sickness benefit offices are responsible for evaluating and monitoring beneficiaries on long-term sickness absence (>30 days) and initiating RTW efforts. Municipal social insurance officers (SIO) assess and classify all sick-listed persons into three categories, distinguishing between beneficiaries who are likely to RTW within three months (category 1), beneficiaries who are not likely to return to work within three months but are able to participate in RTW activities (category 2), and beneficiaries who are not likely to RTW within three months and are not able to participate in RTW activities (category 3).

The first consultation with the SIO about the beneficiaries’ situation and possible steps for RTW must be conducted before the end of the 8th week of absence. Follow-up consultations have to take place every 4th week for category 2 beneficiaries and every 8th week for category 1 and 3 beneficiaries. At the end of the first consultation, the municipal SIO has to develop a plan that includes RTW goals and activities, such as work ability training, gradual return to work, and work modifications. However, the sickness benefit regulations do not specify which kind of activities should be available in a municipality. Initiation of RTW activities depends on the SIO’s assessment of the beneficiary and available municipal resources, which might vary considerably between municipalities (10, 11). In addition to the sickness benefit scheme, municipalities are responsible for the administration of social welfare benefits, vocational rehabilitation, disability benefits, and a wage subsidy scheme. The state partly reimburses municipalities for these benefits. Less reimbursement is paid for passive measures such as sickness and disability benefits. Municipalities therefore have strong incentives to apply an active policy that facilitates labor market reintegration of sick-listed people (9, 12).

Participating municipalities and beneficiaries

The process evaluation was conducted alongside a stratified cluster controlled trial in 21 Danish municipalities (6). Of the 21 municipalities, 13 started the implementation of the RTW program in April 2010, while the remainder served as control group during the first year before implementing the RTW program in April 2011. The original program - scheduled to end March 2012 - was extended until September 2012, but 5 municipalities decided not to continue the implementation during the extended period. Thus, the length of the intervention period varied between one and two and a half years.

There are some slight differences between the 21 municipalities described in the design article (6) and the 21 municipalities described in this article: (i) one control municipality decided to leave the study and was thus not evaluated; (ii) in another municipality, the RTW program was implemented by different RTW teams in two geographically separate sickness benefit offices. We therefore treat this municipality as two different municipalities in the analyses.

The RTW program

The RTW program was integrated into the existing sickness benefit framework and followed the regulations of the ordinary sickness benefit management (OSBM). The detailed content of the RTW program is described elsewhere (6). Briefly, the RTW program consisted of three elements.

Interdisciplinary RTW teams.

Only beneficiaries classified as category 2 were eligible for the study. All participating intervention municipalities were required to establish one interdisciplinary RTW team per 170 category 2 beneficiaries expected to be recruited annually. The interdisciplinary teams consisted of two RTW coordinators (SIO), a psychologist, a physical therapist, a part-time psychiatrist, and a part-time physician of occupational, social, or general medicine.

In contrast to OSBM, RTW coordinators had the possibility to refer beneficiaries to RTW team members for further assessment and clarification of health-related questions. RTW coordinators could also involve RTW team members in RTW activities, eg, in the cooperation with general practitioners and employers. All RTW teams were obliged to conduct weekly team meetings to discuss the situation and necessary activities for beneficiaries who had received or needed further assessment.

The psychologists and physical therapists were asked to establish group education and training sessions on ergonomics training, physical exercises, and psycho-education such as stress and pain management and to offer them to beneficiaries when appropriate. Moreover, depending on the need for further expertise, the RTW coordinators were encouraged to involve the health professionals of the RTW team in other RTW activities eg, meetings with workplaces to discuss workplace modifications and gradual RTW.

Standardized work ability assessment and procedures.

A comprehensive and guided standardized RTW assessment tool was developed, based on a biopsychosocial understanding of health and disability (13), to enable RTW coordinators to conduct a more qualified and systematic assessment of the beneficiaries. RTW coordinators were advised to use the tool in the first consultation with the beneficiary. Furthermore, guidelines and templates for the meetings between the sick-listed beneficiary and the other RTW team members were developed to secure a more qualified and standardized sickness management procedure and support an effective information flow between all RTW team members.

Training courses.

All RTW coordinators, psychologists, and physical therapists were requested to participate in a 3-week training course prior to the onset of the RTW program. Physicians and psychiatrists were requested to participate in preselected parts of the course. Two process consultants from NRCWE visited all RTW teams several times during the intervention period to facilitate the implementation process.

Design of the process evaluation

Inspired by Saunders et al (14), we structured the process evaluation along the following six main topics: (i) reach and recruitment of target population; (ii) recruitment of RTW teams; (iii) fidelity, ie, the extent to which the implementation was consistent with the principles of the interdisciplinary RTW process; (iv) dose-delivered, ie, the extent to which the RTW teams used the program tools, followed the procedures of the program, and cooperated with beneficiaries and external stakeholders; (v) dose-received, ie, the extent to which beneficiaries were exposed to different parts of the RTW program and how beneficiaries assessed the group courses offered by the RTW team members; (vi) contextual factors, ie, aspects that may have influenced the implementation.

To assess these topics, we formulated 29 implementation criteria (IC) presented in table 1.

Table 1

Evaluation topic, implementation criteria (IC), data sources, and analytical approaches. [Quant=quantitative; Qual=qualitative.]

SJWEH-41-529-g001.tif

Data sources

To assess the degree to which the IC were met, we used quantitative and qualitative data from multiple sources. This included municipal administrative records, interviews with RTW team members, field notes from process consultants, and questionnaires or assessments from different groups, including managers of the sickness benefit offices and participants of the group courses offered by RTW team members. The data sources are described in more detail in the appendix (http://www.sjweh.fi/data_repository.php).

To assess reach and recruitment of the target population (IC 1–3, table 1), we used municipal administrative records. We examined how many and what type of beneficiaries (which category) were recruited into the RTW program and whether the first consultation with the RTW coordinator was conducted in time. We calculated the fraction of first consultations before the end of the 9th week of sickness absence (instead of the end of the 8th week) since this is the date the National Labor Market Authority uses when assessing the performance of the Danish sickness benefit offices. Because the first consultation with the RTW coordinator is the only activity that was compulsory for all beneficiaries in the otherwise individually tailored RTW program, this activity is our definition of the beneficiaries participating in the intervention.

To assess recruitment of RTW teams (IC 4), we used semi-structured interviews with RTW teams and field notes from the process consultants.

To assess fidelity (IC 5–13) and dose-delivered (IC 14–20), we developed a rating system inspired by McGrew et al (15). Two data sources were used to independently rate the degree of fidelity and dose-delivered: (i) group interviews with RTW team members in each municipality and (ii) field notes from process consultants (see the Appendix). The research team, which had conducted the group interviews, and the process consultants, who had supported the municipalities in implementing the program, were asked to separately rate to what degree each IC was fulfilled in each municipality using the following rating: 1 (weak adherence to IC), 2 (moderate adherence to IC), and 3 (strong adherence to IC). The research team ratings were based on information given in the group interviews, while the ratings by the two process consultants were based on their insights gathered during their meetings with the RTW teams.

In some cases, data from interviews or process consultants were not sufficient to rate a certain criteria resulting in 28 missing ratings (4.2% of all ratings in the dataset). Seventeen municipalities had either no, one or two missing ratings, while four municipalities had three to five missing ratings. No IC ratings were missing from both research team and process consultants in any municipality. The ratings from the research team and the process consultants came to the same assessment for 32% of IC, similar assessments (difference of ≤1 point in the 3-point rating scale) for 55% of IC, and divergent assessments (difference of >1 point) for 13% of IC. To make use of the ratings from both groups, we combined the two ratings for each municipality and each IC and calculated the mean.

For fidelity, we calculated the average of ratings across IC 5–13 and for dose-delivered, the average of ratings across IC 14–20. For each of these two indexes we defined a total score of ≥2.5 as high fidelity/dose-delivered, a score of 2–2.5 as moderate fidelity/dose-delivered and a score <2 as low fidelity/dose-delivered. In addition, we calculated the fidelity subscales RTW team fidelity (IC 5–11) and leadership fidelity (IC 12–13), and the dose-delivered subscales tools and procedures (IC 14–17) and cooperation (IC 18–20). To assess in more detail to what extent RTW team members had established and executed group trainings for sick-listed beneficiaries in accordance with the RTW program (IC 17), we used data from an assessment conducted among the municipalities.

According to the evaluation model (14), two aspects are relevant when measuring dose-received: exposure and satisfaction. With regard to measuring exposure to the intervention, the tailored design needs to be taken into account. A comprehensive first consultation with the RTW coordinator was supposed to be conducted with all sick-listed beneficiaries participating in the intervention program. Consultations with one or more of the other members of the RTW team were only conducted for those beneficiaries for whom the RTW coordinator decided that a further assessment was needed. For employed beneficiaries, it was recommended that the RTW team should try to establish a close contact with the beneficiaries’ employer, ie, hold at least one meeting at the workplace to discuss work modifications and or gradual RTW. Hence, we present the following four measures of exposure to these different parts of the intervention: (i) the percentage of recruited category 2 beneficiaries who had at least one registered consultation with the RTW coordinator (IC 21); (ii) the percentage of beneficiaries who were referred to further assessment by a psychologist or physical therapist (IC 22) or (iii) by a physician or psychiatrist (IC 23); and (iv) the percentage of beneficiaries for whom a close contact with their workplace had been established (one or more meetings at the workplace, IC 24). We used SIO registrations to assess if at least one consultation with the RTW coordinator had been registered. The other three exposure measures are based on our own data collection on the basis of the municipalities’ registration system for cases referred to the RTW program. The data was collected in the fall of 2011. To guarantee that municipalities had developed a certain routine in their RTW procedures, only the 13 municipalities that started the intervention in the first year (2010) were included in this assessment (see the Appendix). However, as described in the method section, one of these municipalities is treated as two separate municipalities in this article, explaining why 14 results are presented in table 2.

Table 2

Selected measurements of reach and recruitment.

SJWEH-41-529-g002.tif

To measure satisfaction with the intervention, we used two measures of how beneficiaries experienced the group courses offered by the RTW team members (IC 25–26) based on questionnaire data from all participants of group courses during one week in December 2011.

To assess contextual factors, ie, aspects that may have influenced the implementation, we focused on staff and leadership turnover (IC 27–28) and other changes in the municipalities (IC 29). We used data from the questionnaire among the managers of the sickness benefit offices. Managers were asked if turnover in municipal leadership positions and the RTW teams had occurred and, if yes, to comment on the consequences for the implementation. In addition, we asked managers if there had been other changes in the municipality that had a negative impact on the implementation of the RTW program.

Results

Reach and recruitment of target population

Table 2 shows measurements of reach and recruitment of beneficiaries. The percentage of beneficiaries assigned to category 2 and therefore eligible for the RTW program varied greatly between municipalities (22–55%).

It was planned that 14 899 sick-listed beneficiaries would participate in the RTW program. During the project, 12 967 beneficiaries (87%) participated, ie, had at least one consultation with a RTW coordinator. Fifteen municipalities recruited >80% of the planned number of beneficiaries (83–100%), whereas six municipalities recruited <80% (57–77%).

The percentage of first consultations with the RTW coordinator held before the end of the 9th week of sickness absence varied between 5–73%. On average only 50% of the first consultations were held before the end of the 9th week (table 2).

Recruitment of RTW team members

All municipalities recruited the required RTW teams. Not all municipalities completed the recruitment process in time for the training course and the start of the RTW program in the sickness benefit offices. Particularly, the recruitment of psychiatrists took longer in some municipalities.

Fidelity and dose-delivered

The scores for the total fidelity scale varied between 1.2–2.9 (table 3). Five municipalities (24%) reached a score of ≥2.5, indicating a high fidelity, whereas eight municipalities (38%) had a score <2 indicating a low fidelity. The subscales “RTW team fidelity” and “leadership fidelity” were strongly associated, however more municipalities had scores <2 in the subscale “leadership fidelity” (11 municipalities, 52%) than “RTW team fidelity” (five municipalities, 24%).

Table 3

Rating of implementation of fidelity and dose-delivered (total and sub-scales). [IC=implementation criteria.]

SJWEH-41-529-g003.tif

The total scores for dose-delivered varied from 1.5–2.9. Seven municipalities (33%) reached a score of ≥2.5, while only one municipality (5%) had a score <2. Twelve municipalities (57%) obtained scores of ≥2.5 on the subscale “tools and procedures”, while only eight municipalities (38%) obtained scores of ≥2.5 on the subscale “external cooperation”.

Group courses for sick-listed beneficiaries initiated by the RTW team focused, as requested, on physical exercise, mindfulness / relaxation and psychoeducation with regard to stress and pain management, anxiety and depression (IC 17), either as a specific course or in combination with other topics. Between two and six group courses were offered in the different municipalities depending on the size of the municipality and eventually already existing courses. While most RTW teams succeeded in establishing group courses from the start of the project, a few municipalities took up to six months to establish the group courses and the coordination with the already existing courses in the municipalities.

Dose-received

For 88% of all beneficiaries in the intervention group, at least one consultation with the RTW coordinator was registered. This percentage varied from 72–96% between municipalities (table 4). On average 48% of beneficiaries received a further assessment by a psychologist and/or a physical therapist, but this percentage varied between 27–70% among the 14 municipalities where we collected this information (table 4). A further assessment by a physician and/or a psychiatrist was received by 29% of beneficiaries varying between 12–51% among the same 14 municipalities (table 4). At least one meeting with the workplace was held for 9% of all beneficiaries who were employed at the time their sickness absence period started. This percentage varied considerably between the 14 municipalities ranging from 1–23%.

Table 4

Selected measurements of dose-delivered. [NE=not estimated]

SJWEH-41-529-g004.tif

When asked if the group course had helped to get closer to recovery, 38% of the beneficiaries answered “to a very high degree” or “to a high degree”, 34% answered “partly”, 13% answered “to a low degree” or “not at all” and 15% answered “don’t know”. When asked if the courses had helped to better cope with health problems, 55% answered “to a very high degree” or “to a high degree”, 34% answered “partly”, 7% answered “to a low degree” or “not at all”, and 4% answered “don’t know”.

Contextual factors

In all but one municipality, some turnover among the RTW team members occurred during the course of the program (table 5). In total, 14 managers reported that turnovers had no impact on the implementation of the RTW program; 9 managers reported that turnovers had a negative impact and 3 managers reported that turnovers of team members had been positive for the implementation of the RTW program.

Table 5

Turnover in the RTW teams and municipal management positions and other changes in the municipalities that influenced the implementation of the RTW program (assessment based on questionnaire to managers of the RTW program in the sickness benefit offices) +1=positive implication, 0=no implication, -1=somewhat negative implication, -2=negative implication, -3=very negative implication, [NT=no turnover]

SJWEH-41-529-g005.tif

With regard to turnover among municipal leadership positions, (eg, direct supervisor of the RTW program, leader of the municipal sickness benefit office, leader of the municipal employment center), three managers answered that turnovers had a negative impact while three other managers pointed out that the turnovers had positive implications for the implementation of the RTW program. In the remaining municipalities, managers reported that there was no turnover or that the turnover had little or no impact on the implementation.

Four managers reported that other changes in the municipalities, ie, location or relocation of the RTW team, management turnover in the health department of the municipality and organizational changes in the municipality had a negative effect on the implementation of the RTW program.

Discussion

The evaluation of the implementation of the RTW program can be summarized into two main findings. First, all municipalities were able to integrate the basic features of the RTW program into the existing framework of the sickness benefit management system. Second, for most implementation aspects, we found large variations between the municipalities.

Integration of RTW program into existing framework

In relation to integration of the RTW program into the existing framework, two aspects deserve further discussion. Firstly, in some municipalities, the recruitment of team members, in particular psychiatrists, was not always timely with respect to the training course. Difficulties in recruiting psychiatrists are probably due to the shortage of psychiatrists in Denmark (16). Thus, shortage of psychiatrists with knowledge of the work participation of beneficiaries with mental health problems may be a drawback in future RTW activities.

Secondly, the percentage of beneficiaries assigned to category 2 varied considerably between municipalities, despite that the definition for the three categories was based on pre-existing regulations applying to all Danish municipalities. Some of the variation could be due to socioeconomic or demographic differences in the municipal populations, eg, a higher percentage of older persons leading to a higher percentage of category 3 cases. Moreover, the definition of the three categories was altered by the National Labor Market Authority close to the start of the intervention, leading to temporary uncertainties among the municipal administrators. Nevertheless, the large variation suggests that the administrative personnel responsible for assessing the beneficiaries differed in their interpretation of the categorization criteria. Consequently, different interpretations of the categorization criteria by the municipalities may have resulted in systematic differences in participant characteristics between municipalities. The use of pre-existing regulations applying to all Danish municipalities thus proved to be insufficient for guaranteeing a satisfactory selection of similar participants into the program across municipalities.

Large implementation variation between municipalities

Large variations in implementation were found for most measures of fidelity, dose-delivered, and dose-received. This is in line with a recently published article about the implementation of a multidisciplinary intervention for beneficiaries with mental health problems. Due to different barriers and facilitators, the quality of implementation varied greatly across the three participating municipalities resulting into different versions of the intervention (17). Large variations might partly be due to the complexity of the intervention. Complex interventions need to allow variation in their delivery, making these kinds of interventions more vulnerable to one or more components not being implemented as they should (18). In our study, we define the underlying principles of the interdisciplinary RTW process as interdisciplinary teamwork that includes mutual learning and continuous evaluation to achieve a more in-depth and more precise work ability assessment that is supported and guided by management. Considering that the implementation of interdisciplinary teams within the context of Danish municipal sickness benefit offices was a novelty for almost all RTW team members, it might not be surprising that almost 40% of the municipalities complied with these principles only to a low degree, indicating that interdisciplinary teamwork was particularly difficult to achieve. Research in this area highlight the efforts needed to achieve well-functioning interdisciplinary teams: importance of consistent team membership to foster strong relationships and group identity (19), enough time and practice (20), input from the organizational level (21), and trying to unite the different perspectives of the various RTW stakeholders (17, 2224).

The dose-delivered results suggest that the RTW teams in general were able to use the tools developed for a more standardized and qualified assessment as well as information flow and generally followed the procedures of the program. In contrast, the cooperation between RTW teams and workplaces, general practitioners, and the sick-listed beneficiaries was more difficult to achieve.

More than half of the beneficiaries (55%) reported that the RTW teams’ group courses had helped them to better cope with their health problems and more than a third (38%) answered that the group courses had positive effects on their recovery. These evaluations from beneficiaries are in line with a qualitative study of experiences with the RTW program among beneficiaries with self-reported stress and depression (25). The interviewed participants evaluated psychoeducative group courses as positive and pointed out that the courses helped them to normalize their condition, restored their self-confidence, and reduced their feeling of being alone. However, the study also found that work ability assessment consultations could result in both motivation and frustration. Beneficiaries’ experiences depended on the extent to which RTW team members were able to practice an “individual approach”, ie, if they treated and respected beneficiaries as unique persons with specific problems and showed genuine interest in their situation and their needs in the RTW process (25). This indicates that beneficiaries’ experiences are not only influenced by which RTW activities are offered to them but also largely by how these activities are presented and conducted.

When planning the study, we anticipated that about 50% of beneficiaries would need further assessment by a psychologist and/or a physical therapist and about 25% would need further assessment by a physician and/or a psychiatrist. While the average rates of referrals across all municipalities were very close to what we anticipated, referral rates varied considerably between municipalities. Since the referral to other RTW team members was supposed to be based on the need for further assessment, no goals or recommendations were set beforehand. However, a very low referral rate might indicate that the expertise of the other RTW team members was not always used, while a very high referral rate might indicate an overuse of the other RTW team members’ expertise.

The proportion of beneficiaries having their first consultation with the RTW coordinator before the end of the 9th week ranged from 5–73% (table 2). Similar results have been found in a recent study in five municipalities conducted by the Danish National Audit Office (26). Thus, the delay in first consultation seems to be a general problem in the Danish sickness benefit system and not specific for this intervention. Among the possible reasons for this delay are: (i) workplaces and beneficiaries have too weak incentives for timely report of sickness absence episodes and (ii) lack of efficiently organized cooperation within the municipality (26).

Because employees in Denmark have little protection from dismissal when sick-listed (12), the late involvement of the municipalities may be especially disadvantageous, as chances for establishing a close contact with employers and developing collaborative RTW plans may be reduced. An earlier study among employees sick-listed for >8 weeks found that >50% had been dismissed or had quit (27). Therefore, it is very likely that a large percentage of beneficiaries was not employed anymore by the time the sickness benefit offices were ready to establish a close contact with the beneficiaries’ employer. Also other reasons for not initiating a close cooperation with the employer need to be considered: a sick-listed beneficiary and his/her employer might already have agreed on how to organize the RTW process, or the sick-listed beneficiary did not want to establish a close contact with his/her employer because of existing conflicts. These obstacles might explain the low number of workplace meetings (9% on average for all beneficiaries who were employed at the time their sickness absence period started). Underreporting due to inconsistent registration might also play a role. However, our results are in line with earlier studies that pointed out that close contacts between sickness benefit offices and employers are often missing (26, 28). Considering these obstacles, high rates of close contacts with employers might be difficult to achieve in Denmark. However, resources might also play a role. While managers of sickness benefit offices and SIO agreed that meetings between SIO, the beneficiary and the beneficiaries’ employer were very useful, they also pointed out that they required a lot of time which can be difficult to find while having to work on a large number of cases at the same time. Fewer cases per SIO might therefore help to increase the number of cooperations with workplaces. Although this initially will require more resources, it might turn out to be more effective in the end, as studies consistently have found that interventions involving the workplace, to – for example – implement work modifications, were more effective than interventions not involving the workplace (29, 30). The most recent change in the Danish sickness benefit scheme, which took effect in 2014, addresses this and other problems. The new regulations include a “Fast track” solution for employees and employers who like to involve the municipal job centers at an earlier stage in cases of sickness absence that are expected to last >8 weeks.

Contextual factors

Almost all municipalities experienced some staff turnover or other changes in the municipality or both, and several managers reported that these changes had a negative impact on the implementation of the RTW program. It can be discussed if it is justified to classify turnover within the RTW teams as a contextual factor as it also can be a reaction to problems within the RTW team. While this might be the case, each turnover within the team is a disturbance and contains therefore also elements of a contextual factor with which the remaining team has to deal.

Most municipalities seem to have been able to handle the contextual changes we assessed in our study, because no particular pattern was seen between contextual changes and problems in the implementation of the different elements of the RTW program. Yet, the ability to handle contextual changes might reach a limit when too many disturbances occur at one time. In the two municipalities where managers reported negative or very negative implications caused by staff turnover in addition to negative implications due to other contextual changes, we also found low ratings for fidelity. The development of a well-functioning interdisciplinary RTW team takes time and requires a certain continuity of team members (19, 20). Staff turnover and other changes that negatively influence the cooperation within the RTW team disturb this development. Furthermore, new team members had not participated in the training course prior to the start of the intervention and might therefore have had more difficulties in understanding the aims and procedures in the RTW program.

Still, in a few other municipalities managers reported that staff turnover had a positive effect on the program implementation. This might indicate that some managers during the course of the intervention became more aware of the competencies needed and subsequently recruited RTW team members that better fitted to the interdisciplinary approach of the RTW program. Only four managers answered that other changes in the municipalities had a negative effect on the program implementation. Thus, although other contextual changes might have affected all municipalities (changes in sickness benefit regulations or the economic recession making RTW more difficult), most managers did not consider problems in implementing the RTW program to be due to contextual changes within their municipality.

Strength and weaknesses of the study

The strength of this study is the comprehensive and structured process evaluation using qualitative and quantitative data gathering methods and quantitative analysis for assessing 29 implementation criteria. Furthermore, we developed a rating system that allowed us to systematically assess whether and to what extent the different aspects of fidelity and dose-delivered had been implemented. This approach allowed us to identify which aspects of the RTW program were particularly difficult to implement.

The use of a large part of our qualitative data in such a quantitative way could, however, also be regarded as a weakness, as it did not allow us to study the quality of the implementation in depth and limited the opportunity to understand why implementation was or was not successful (31).

It would have been desirable to have more (and better) measures of beneficiaries’ exposure to the different program activities. That would have enabled us to study in more detail who received the different components of the program and how they were perceived by beneficiaries. Two attempts were made to document these activities. First, we asked the RTW coordinators to register all main activities for each beneficiary. However, the registrations were done very inconsistently due to too many already existing administrative registration tasks. Second, we tried to document the different program activities based on the municipalities’ own case records. However, also this attempt turned out to be largely impaired by the different ways of recording in each municipality and by each SIO (see Appendix under administrative data). Thus, we obtained very few measures to assess beneficiaries’ exposure to the program, which limits a more detailed evaluation of how the different program components were experienced by beneficiaries.

Finally, our assessment of contextual factors that was confined to staff and leadership turnover and other changes in the municipality might have been too narrow. Broader contextual aspects of the municipalities such as socioeconomic and demographic characteristics of the municipalities’ residents, local unemployment rate, and municipal budget might also have played a role for the implementation of the RTW program but were not assessed in the process evaluation. However, these aspects might be more important when assessing the effects of the intervention.

Concluding remarks

Although it was feasible to integrate the basic features of the Danish RTW program, large variations existed between municipalities for most of the implementation criteria. Establishment of well-functioning interdisciplinary RTW teams might require more time and resources, while ensuring early assessment and more frequent cooperation with employers might need more general adjustments in the Danish sickness benefit system. Recent changes in the Danish sickness benefit scheme have tried to address these issues. However, it remains to be seen if these changes will lead to the desired improvements.

Note

In a Danish report prepared immediately after the end of the intervention (32), we combined all of the ratings used in this study into one score. For this article, however, we conducted a more detailed analysis and, as part of this, divided the ratings into those presenting fidelity and those presenting dose-delivered according to the concept introduced by Saunders et al (14).

Acknowledgements

The RTW program was funded by the Danish National Prevention Fund to reimburse the municipalities for the extra costs of case management for beneficiaries receiving the program. In addition the Ministry of Employment financed the evaluation of the program.

References

1 

Loisel, P, Buchbinder, R, Hazard, R, Keller, R, Scheel, I, van, TM, & Webster, B. (2005). Prevention of work disability due to musculoskeletal disorders: the challenge of implementing evidence. J Occup Rehabil, 15, 507-24, http://dx.doi.org/10.1007/s10926-005-8031-2.

2 

Loisel, P, Anema, JR, Loisel, P, & Anema, J. R (Eds.). (2013). Handbook of Work Disability, Prevention and Management, New York, Heidelberg, Dordrecht, London, Springer, http://dx.doi.org/10.1007/978-1-4614-6214-9.

3 

Craig, P, Dieppe, P, Macintyre, S, Michie, S, Nazareth, I, & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. British Medical Journal, 337, http://dx.doi.org/10.1136/bmj.a1655.

4 

Craig, P, & Petticrew, M. (2013). Developing and evaluating complex interventions: Reflections on the 2008 MRC guidance. International Journal of Nursing Studies, 50, 585-7, http://dx.doi.org/10.1016/j.ijnurstu.2012.09.009.

5 

Moore, GF, Audrey, S, Barker, M, Bond, L, Bonell, C, Hardeman, W, Moore, L, O'Cathain, A, Tinati, T, Wight, D, & Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ, 350, 1258, http://dx.doi.org/10.1136/bmj.h1258.

6 

Aust, B, Helverskov, T, Nielsen, MB, Bjorner, JB, Rugulies, R, Nielsen, K, et al. (2012). The Danish national return-to-work program--aims, content, and design of the process and effect evaluation. Scand J Work Environ Health, 38, 120-33, http://dx.doi.org/10.5271/sjweh.3272.

7 

Poulsen, OM, Aust, B, Bjorner, JB, Rugulies, R, Hansen, JV, Tverborgvik, T, et al. (2013). Effect of the Danish return-to-work program on long-term sickness absence: results from a randomized controlled trial in three municipalities. Scand J Work Environ Health, 40, 47-56, http://dx.doi.org/10.5271/sjweh.3383.

8 

Nielsen, MB, Hansen, JV, Aust, B, Tverborgvik, T, Thomsen, BL, Bjorner, JB, et al. (2014). A multisite randomized controlled trial on time to self-support among sickness absence beneficiaries. The Danish national return-to-work programme. Eur J Public Health, 25, 96-102, http://dx.doi.org/10.1093/eurpub/cku016.

9 

Johansen, K, Andersen, JS, Mikkelsen, S, Pass, O, Raffnsoe, S, & Lynge, E. (2008). Controlling sickness absence: a study of changes in the Danish sickness absence legislation since 1973. Health Policy, 86, 109-18, http://dx.doi.org/10.1016/j.healthpol.2007.10.006.

10 

Høgelund, J, Boll, J, Schou, M, & Jensen, S. (2008). Effekter af ændringerne i sygedagpengeloven. Opfølgning påsygedagpenge [Effects of changes in the public sickness benefit scheme. Follow-up on sickness benefits]. København: SFI- Det Nationale Forskningscenter for Velfærd.

11 

Stoltenberg, CD, & Skov, PG. (2010). Determinants of return to work after long-term sickness absence in six Danish municipalities. Scand J Public Health, 38, 299-308, http://dx.doi.org/10.1177/1403494809357095.

12 

Høgelund, J, & Holm, A. (2006). Case management interviews and the return to work of disabled employees. J Health Econ, 25, 500-19, http://dx.doi.org/10.1016/j.jhealeco.2005.07.007.

13 

Loisel, P, Gosselin, L, Durand, P, Lemaire, J, Poitras, S, & Abenhaim, L. (2001). Implementation of a participatory ergonomics program in the rehabilitation of workers suffering from subacute back pain. Appl Ergon, 32, 53-60, http://dx.doi.org/10.1016/S0003-6870(00)00038-7.

14 

Saunders, RP, Evans, MH, & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Prac, 6, 134-47, http://dx.doi.org/10.1177/1524839904273387.

15 

McGrew, JH, Bond, GR, Dietzen, L, & Salyers, M. (1994). Measuring the fidelity of implementation of a mental health program model. J Consult Clin Psychol, 62, 670-8, http://dx.doi.org/10.1037/0022-006X.62.4.670.

16 

Danish Psychiatric Society. (2011). Hvidbog 2012-2020 [White-paper 2012-2020]. Copenhagen: Dansk Psykiatrisk Selskab.

17 

Martin, MH, Moefelt, L, Nielsen, MB, & Rugulies, R. (2015). Barriers and facilitators for implementation of a return-to-work intervention for sickness absence beneficiaries with mental health problems: Results from three Danish municipalities. Scand J Public Health, 43, 423-31, http://dx.doi.org/10.1177/1403494814568484.

18 

Carroll, C, Patterson, M, Wood, S, Booth, A, Rick, J, & Balain, S. (2007). A conceptual framework for implementation fidelity. Implement Sci, 2, 40, http://dx.doi.org/10.1186/1748-5908-2-40.

19 

Sinclair, LB, Lingard, LA, & Mohabeer, RN. (2009). What's so great about rehabilitation teams? An ethnographic study of interprofessional collaboration in a rehabilitation unit. Arch Phys Med Rehabil, 90, 1196-201, http://dx.doi.org/10.1016/j.apmr.2009.01.021.

20 

Shaw, L, Walker, R, & Hogue, A. (2008). The art and science of teamwork: enacting a transdisciplinary approach in work rehabilitation. Work, 30, 297-306.

21 

Cartmill, C, Soklaridis, S, & David, CJ. (2011). Transdisciplinary teamwork: the experience of clinicians at a functional restoration program. J Occup Rehabil, 21, 1-8, http://dx.doi.org/10.1007/s10926-010-9247-3.

22 

Young, AE, Wasiak, R, Roessler, RT, McPherson, KM, Anema, JR, & van Poppel, MNM. (2005). Return-to-work outcomes following work disability: Stakeholder motivations, interests and concerns. J Occup Rehabil, 15, 543-56, http://dx.doi.org/10.1007/s10926-005-8033-0.

23 

Ståhl, C, Svensson, T, Petersson, G, & Ekberg, K. (2010). A Matter of Trust? A Study of Coordination of Swedish Stakeholders in Return-to-Work. J Occup Rehabil, 20, 299-310, http://dx.doi.org/10.1007/s10926-009-9205-0.

24 

MacEachen, E, Kosny, A, Ferrier, S, & Chambers, L. (2010). The “toxic dose” of system problems: why some injured workers don't return to work as expected. J Occup Rehabil, 20, 349-66, http://dx.doi.org/10.1007/s10926-010-9229-5.

25 

Andersen, MF, Nielsen, K, & Brinkmann, S. (2014). How do Workers with Common Mental Disorders Experience a Multidisciplinary Return-to-Work Intervention? A Qualitative Study. J Occup Rehabil, 24(4), 709-24, http://dx.doi.org/10.1007/s10926-014-9498-5.

26 

The Danish National Audit Office. Beretning til Statsrevisorerne om indsatsen for at fåsygemeldte tilbage i arbejde [Report to the Public Accounts Committee on the effort made to help people on sick leave return to work]. (2014). Copenhagen: Rigsrevisione.

27 

Høgelund, J, Filges, T, & Jensen, S. (2003). Langvarigt sygefravær - hvad sker der, og hvordan går det?[Long-term sickness absence - what happens and how does it turn out?]. København: Socialforskningsinstituttet.

28 

Johansen, K, Andersen, JS, Mikkelsen, S, & Lynge, E. (2011). Decision making and co-operation between stakeholders within the process of sick leave. A case study in a Danish municipality. J Interprof Care, 25, 59-65, http://dx.doi.org/10.3109/13561820.2010.483367.

29 

Carroll, C, Rick, J, Pilgrim, H, Cameron, J, & Hillage, J. (2010). Workplace involvement improves return to work rates among employees with back pain on long-term sick leave: a systematic review of the effectiveness and cost-effectiveness of interventions. Disabil Rehabil, 32, 607-21, http://dx.doi.org/10.3109/09638280903186301.

30 

Gabbay, M, Taylor, L, Sheppard, L, Hillage, J, Bambra, C, Ford, F, Preece, R, Taske, N, & Kelly, MP. (2011). NICE guidance on long-term sickness and incapacity. Br J Gen Pract, 61, e118-e124, http://dx.doi.org/10.3399/bjgp11X561221.

31 

Egan, M, Bambra, C, Petticrew, M, & Whitehead, M. (2009). Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health, 63, 4-11, http://dx.doi.org/10.1136/jech.2007.071233.

32 

The National Research Centre for the Working Environment. Det Store TTA-projekt. Process-, effekt- og økonomisk evaluering [The large RTW project. Process-, effect- and economic evaluation]. (2012). Copenhagen: Det Nationale Forskningscenter for Arbejdsmiljø.


Additional material
  • download Data sources used for the process evaluation of the RTW program