Return to work and occupational physicians’ management of common mental health problems – process evaluation of a randomized controlled trial

Objective The aim of this study was to examine the adherence of occupational physicians (OP) to the Dutch guideline on the management of common mental health problems and its effect on return to work as part of the process evaluation of a trial comparing adherence to the guideline to care as usual. The first hypothesis was that guideline adherence among the “guideline group” will be higher compared to the “usual care group”. The second hypothesis was that better guideline adherence by the occupational physician will be associated with earlier return to work. Methods In a randomized controlled trial, five participating OP had to provide care based on the Dutch guideline to 240 police workers with common mental health problems (the “guideline group”). The same OP had to provide usual care to the participants in the control group (the “usual care” group), including minimal involvement and easy access to a psychologist. In evaluating the process, we assessed guideline adherence via an audit of medical files, using 20 guideline-based performance indicators. Mean rates of guideline adherence were related to the duration until first and full return to work, using a Cox proportional hazards model. Results The mean rate of the sum score of guideline adherence was 10 in a range of 0–20 [standard deviation (SD) 1.8] and did not significantly differ between the intervention and control group. Mean better guideline adherence showed a statistically significant association with a shorter time to first and full return to work [hazard ratio 1.1; 95% confidence interval (95% CI) 1.0–1.2], which was explained by keeping more regular contact with the worker and the work system and better monitoring of stagnation or return to work. Conclusions No contrast in guideline adherence was found between guideline-based versus usual care. This can be explained by contamination between the guideline and usual care group. Even though guideline adherence was only average, better adherence predicted earlier return to work. Guidelines for management of common mental health problems and return to work should focus on regular contact with the worker and the work organisation.

Common mental health problems among workers (eg, adjustment disorders, depression, and anxiety) increasingly affect functioning to such an extent that they lead to sick leave, early retirement, and productivity loss (1,2). Primary and occupational healthcare of workers with common mental health problems usually focus on recovery of symptoms instead of return to work (3). In the Netherlands, workers have to visit an occupational physician (OP) for return-to-work purposes when they are on sick leave (4). Therefore, OP play a central role in the treatment of workers with common mental health problems.
In 2000, the Netherlands Society of Occupational Medicine (NVAB) published a guideline entitled "Management of common mental health problems by occupational physicians" (4,5). This guideline promotes a more active role of the OP as a counselor facilitating a worker's return to work compared to usual care, in which the OP usually has a minimal role. In this minimal role, the OP often refers the worker to a psychologist Rebergen et al if complaints persist (6). This may lead to a "referral delay", which is often associated with a delayed return to work (7)(8)(9). Consequently, workers may not get the optimal care they need.
We conducted a randomized controlled trial (RCT) with a 12-month follow-up to evaluate the effectiveness of the OP guideline compared to usual care (10). The use of the guideline did not result in better return-towork outcomes (11). However, the use of the guideline appeared to be more cost-effective, as healthcare costs were lower in the "guideline group" compared to the relatively higher costs of psychological treatment in the "usual care group" (12). To determine the actual use of the guideline, we performed a process evaluation in which OP adherence to the guideline was assessed and related to return to work (13)(14)(15). Although other studies have presented data on the relationship between guideline adherence and return to work, no firm relation has been established (3,9,(16)(17)(18)(19)(20) These studies used a set of performance indicators (PI) to measure guideline adherence, interpreted as markers of quality of care.
The aim of this study was to examine the adherence of Dutch OP to the guideline and its effect on return to work as part of the process evaluation of a trial comparing adherence to the guideline to care as usual. The first hypothesis was that guideline adherence among the guideline group will be higher compared to the usual care group. The second hypothesis was that better guideline adherence by the occupational physician will be associated with earlier return to work.

Design
The RCT evaluated the effect of the Dutch national guideline on OP management of workers with common mental health problems (10). Subjects in the intervention or guideline group were treated by OP trained to provide guideline-based care. The control or usual care group received usual care by the same OP with his or her minimal involvement and, if applicable, easy access to treatment by a psychologist. All the participating OP received the guideline training and had to provide both guideline and usual care, risking treatment contamination. Participants, employers, and OP were not blinded for the intervention. Researchers were blinded for treatment allocation and protocol compliance.
The Medical Ethics Review Committee of the VU University Medical Centre approved the study design, protocol, and procedures. Details of the study design and the guideline have been reported elsewhere (10). A brief summary is presented below.

Study population
This study was conducted with the cooperation of the Dutch police force, which is an organization with a relatively high incidence of stress-related sick leave (21). Two police departments, comprising a source population of 2500 police workers, were chosen because they had contracts with the same occupational health service. The study commenced in 2002, after which time police workers were included upon consultation with an OP for sick leave due to common mental disorders. Between January 2002 and January 2005, the occupational health service registered 489 workers as being absent from work due to common mental health problems (figure 1) (11). Of those, 240 workers met the inclusion criteria and signed an informed consent form. Randomization into two groups (the guideline and usual care groups) was done at the individual level of participants. Of the 240 subjects, 125 subjects were assigned to the guideline group and 115 subjects were assigned to usual care. Although not all the questionnaires were returned by the subjects, nobody was lost to follow-up regarding guideline adherence as performance rates were based on the medical files. The return-to-work data of workers who left their jobs or died (N=16), or did not return to work during the one-year follow-up (N=24), were censored.

Participating occupational physicians
Five OP participated in this study. In the case of the smallest police department (800 workers), the same OP worked continuously during the study. The largest police department (2500 workers) changed occupational health service after one year. Therefore, a new occupational health service participated in the study with two different OP, one of them being replaced by another new OP in the third year of the study. Thus, finally five OP, two men and three women, participated in the study.

Usual care
Usual care consisted of minimal involvement of the OP and easy access to a psychologist, whose treatment was mainly based on cognitive behavioral therapy principles. This psychological treatment was fully funded by the health insurance company of the Dutch police, and OP were, therefore, facilitated to refer patients in the usual care group to a psychologist. In this way, usual care was psychologist-based and enriched with respect to usual care in other countries.

Intervention
In the guideline group, the intervention consisted of treatment by an OP according to the Dutch guideline Return to work and physician's management of mental problems on how to manage workers with common mental health problems (4,5,10). The guideline focuses on five constructs in the management of common mental health problems. First, a problem orientation takes place, in which the OP acknowledges the interaction between the disabled worker and his surroundings (work, personal, and care). Second, a simplified classification of common mental health problems is introduced, with only four categories: (i) adjustment disorder (distress, nervous breakdown, burnout); (ii) depression; (iii) anxiety; and (iv) other psychiatric disorders. Third, the OP is encouraged to take early and activating interventions, in which time-contingent return to work is part of the recovery process, even if the common mental health problems are not related to work. The guideline provides activating homework assignments that the OP can use, for example, to initiate the worker to start, structure, and monitor their activities. The OP is encouraged to act as a counselor, applying cognitive behavioral techniques to workers with stress and/or work-related problems. Early and activating interventions could also include treatment by or referral to a company social worker and/or psychologist as these interventions could also be conducted by other professionals. Fourth, a timecontingent evaluation is done, in which the OP acts as case manager who intervenes when recovery stagnates. Finally, relapse prevention is an integral aspect of the treatment, in which the worker is encouraged to undergo a risk analysis and follow a related action plan.
Participating OP received training in the guideline before the study started. During this three-day educational course, 10-15 OP were trained in the use of the guideline (22). The worker's own responsibility in the recovery process was emphasized, as was the importance of starting the intervention early with the aim of acquiring coping skills and regaining control. The OP were encouraged to use specific tools, such as the four-dimensional symptom questionnaire (4DSQ) (23), patient information leaflets on stress, and day-structuring exercises. The OP were free to choose specific tools in each phase of the process.

Guideline adherence
To study our first hypothesis, we assessed the OP adherence to the guideline as a primary outcome using PI. These indicators were rated by means of an audit of the medical records of the occupational health service. Smits et al (16) developed an initial set of ten guideline-based PI and corresponding criteria, which were validated by Nieuwenhuijsen et al (17). This validated set evaluated a substantial part of the guideline regarding problem orientation, diagnosis, and interventions. As the indicators and criteria did not cover the complete treatment process, we developed ten additional PI and added these to the initial set. These additional indicators also focused on the OP's counseling role, progress evaluation, and relapse prevention. They were based on recent scientific literature (3, 9, 16-20, 24, 25). The extended set of 20 PI was categorized according to the five core Workers registered as being absent from work due to mental health problems (N=489) Inclusion in randomized controlled trial during consultation occupational physician in which informed consent was obtained (N=240)

Figure 1 -Flow chart CO-OP study, a randomized controlled trial on guideline-based care by occupational physicians of workers with mental health problems
Return to work (sick leave days) Return to work (sick leave days) Guideline adherence Guideline adherence  (table 1). The initial set of ten indicators consisted of PI 1-5, 7, 9, and 15-17. For each PI, criteria were used that were based on an if-then logic. For example, IF the treatment in curative care is not effective, THEN the OP should contact the practitioner. If criteria of a PI were met or not applicable, the score for that indicator was 1 corresponding with maximal guideline adherence. If criteria were not met, the score was 0. In this way, an average performance rate was obtained for each PI. A higher performance rate reflects higher guideline adherence on the specific related topic.
Guideline adherence for the complete treatment process was measured with the extended set of 20 PI as the mean PI score divided by the total number of PI.

Assessment of guideline adherence
Two researchers independently assessed the PI using the medical records of the occupational health services. If the ratings were not congruent, a third researcher was bound to make a decision. PI were calculated using the participant's medical records data, gathered from the databases of the participating occupational health services. To guarantee blinding of the outcome assessors, the data of the medical records were stripped of non-essential information and were transferred to a Microsoft Access database (Microsoft Corporation, Redmond, MA, USA). A digital score list was created in the program SPSS Data Entry Builder version 3.0 (SPSS Inc, Chicago, IL, USA), in which individual criteria of the PI could be scored. The criteria of the PI were converted into scores using SPSS syntaxes, enabling computerized appraisal of guideline adherence.

Return to work
To address the second hypothesis, first and full return to work were defined as the duration in sick leave due to common mental health problems from the moment of inclusion until first (partial or full) and full return to work, respectively. This was return to work in the same job as before the onset of the sick leave or a job with equal earnings (26,27). Sick leave data were gathered from records of the police departments.

Potential confounders
Confounders were defined as variables that could potentially influence the relation between the OP's adherence to the guideline and the worker's return to work. The following personal, work, and treatment characteristics were measured as potential confounders: gender, age, children in family (yes/no), OP, occupational health service, contract hours per week, type of work (executive or administrative), sick leave days in the previous year, work-relatedness of the disorder, and severity of symptoms. These data were gathered from the records of the police departments and the medical files of the occupational health services. Severity of depression, anxiety and stress was measured at baseline, using the 42-item depression anxiety stress scale (DASS-42) and the hospital anxiety depression score (HADS) (28,29).

Statistical analysis
Due to the nonparametric nature of the data, we calculated Spearman rank correlations to assess the interrelations between the PI. A rule of thumb was applied, stating that at least one intercorrelation of ≥0.70 would suggest colinearity and lead to the exclusion of one of these variables (17). We compared mean performance rates on guideline adherence of both groups using independent T-tests. This was done for each PI separately and for the sum score. Only those PI that showed sufficient variability (<90% of patients with the same score) were selected for further analysis.
Our second hypothesis was that better OP adherence to the guideline is associated with earlier return to work. To investigate this, we used multivariate Cox regression analyses to relate the mean sum score of the PI and the separate PI to the time to first and full return to work. All PI that showed sufficient variability (<90% of patients with the same score) were selected for these analyses (17). We added the potential confounders one by one and tested whether this led to a change in one of the estimates of the independent variables of >10%. If so, the potential confounder that generated the largest overall change in betas was added to the model, after which the other potential confounders were once again added to the model, one by one. This procedure was repeated until the addition of potential confounders caused no substantial change in any estimate of an independent variable. All variables were entered into the model in a single step (method: enter). Statistical analyses were performed using SPSS version 15.0 software package (SPSS Inc, Chicago, IL, USA).

Guideline adherence
Baseline data, treatment and return-to-work characteristics are shown in table 2. In the usual care group, subjects received significantly more psychological treatments; in the guideline group, there were significantly more treatments by company social workers. Although only 17% in the guideline group were immediately Return to work and physician's management of mental problems  PI 20 Relapse prevention by the OP Criterion: After full RtW the OP has at least 1 consultation with the patient Criterion should be met for guideline adherence a Initial ten performance indicator-set by Nieuwenhuijsen et al (17). Rebergen et al referred to a psychologist, 46% received psychological treatment from a psychologist and/or psychiatrist during the one-year follow-up period. In the usual care group, 82% of the participants were immediately referred to a "funded" psychologist, representing almost all of the 85% receiving treatment from a psychologist and/or psychiatrist during the one-year follow-up period. Table 3 shows the mean performance rates for both the guideline and usual care groups. No significant differences were found between the groups on most performance rates. There was a significantly higher performance rate on the use of the 4DSQ in the guideline group to assist in the classification of the mental health problem (22% versus 10%, P=0.02). Early and activating interventions aimed at individual workers, which included referral to a psychologist or social worker differed significantly between the guideline and usual care groups (80% versus 93%, P<0.01). In the guideline group, the OP contacted the general practitioner more often (7% versus 1%, P<0.01).

Guideline adherence and return to work
The results of the Cox's regression analyses are shown in table 4. As adequate guideline adherence is indicated with 1.0 and poor guideline adherence with 0, a hazard Return to work and physician's management of mental problems ratio (HR) >1 indicates the risk of a shorter time to return to work for the group with adequate compared to poor guideline adherence. For example, a HR of 1.5 means that the adequate guideline adherence group returned to work two times faster compared with the poor guideline adherence group. The PI 12-15 were excluded from these analyses as a result of lack of variability. Therefore, 16 PI could be used in the analyses. Three PI regarding evaluation of progress (PI 16, 17, and 19) were significantly related to a shorter time to first and full return to work (table 4). Thus workers, whose OP had regular consultations with the worker and the work context and intervened on stagnation of return to work, returned to work earlier.
The sum score revealed that more adequate guideline adherence showed a statistically significant association with a shorter time to first and full return to work [HR 1.1, 95% confidence interval (95% CI), 1.0-1.2]. This model was adjusted for the confounders OP, HADS total score, children, number of sick leave periods in the previous year, work relatedness, and type of function.

Guideline adherence
The mean performance rate did not differ significantly between the guideline and usual care groups (50% in both groups). The most reasonable explanation for the lack of contrast between the groups may be the fact that all five participating OP were trained in the guideline and had to provide both guideline and usual care to the participants. Additionally, no activities were undertaken to improve the OP's treatment compliance with the allocated treatment.
Because of this risk of treatment contamination, we tried to maximize the contrast between guideline and usual care by creating a situation in which referral to a psychologist in usual care was always granted by the Dutch police force's health insurance company. The intention was that this would address a common obstacle (ie, the time it takes to refer the worker to another a professional) and encourage the OP to play Rebergen et al a minimal role. Psychological treatments were indeed significantly higher in usual care, but referral patterns to a company social worker were significantly higher in the guideline group. This caused an unexpected lack of contrast between the groups, as the intended and expected involvement of the OP in the guideline group did not seem to materialize, as no differences in guideline adherence were found between the groups. Another possible explanation for the lack of contrast in guideline adherence between the groups may be the nature of the guideline itself. Many practice guidelines, including this guideline, are partially practice-based. This may explain the 50% guideline adherence which was found in the usual care group. This percentage is comparable to findings from other studies (3,9,(16)(17)(18)(19)(20). Another explanation for the lack of contrast in guideline adherence may be the introduction of the Gatekeeper Improvement Act in 2004, which obliged OP to have regular contact with the worker and his/her employer.

Guideline adherence and return to work
Better guideline adherence showed a statistically significant association with a shorter time to first and full return to work. Workers whose OP had regular consultations with the worker and the work context and intervened on stagnation of the worker returned to work earlier. The results of adequate guideline adherence indicate that the OP treatment should focus on these elements to shorten the work absence of workers with common mental health problems. Due to a lack of variability, potential important indicators were not taken into account into the model or only with a weak contribution. Among these elements were: (i) the assessment of symptoms, (ii) the use of the 4DSQ, (iii) inquiring about the expectations of patients on return to work, (iv) OP counseling of patients, (v) OP contact with the general practitioner, and (vi) OP effort to prevent relapse after return to work. The infrequent use of these essential elements of the guideline may be the result of difficulties faced by the OP in implementing this guideline. Consequently, the relationship between these guideline adherence indicators and return to work could not be analyzed.

Strengths and limitations of this study
In this study, guideline adherence was assessed by an audit of medical files, which is an indirect method to measure clinical performance (30). This has the advantage that the OP were unaware that their notes would be used for research purposes. However, an audit of medical records is, as any observational study, susceptible to bias (31).
A first possible source of bias is that the OP did not register all their findings systematically for use in research. Exceptions were the frequency of the OP consultations with both the worker and the employer, as the occupational health service routinely listed these data, ruling out the possibility of inaccurate registration. A second source of bias is related to this last notion, as the two participating occupational health services used different registration systems, which may have influenced performance rates. A third specific source of bias in this study may have been the circumstance that the largest participating police department changed occupational health service during the course of our study. The adaptation to this new situation may have had a negative influence on treatment in the guideline group. In usual care, these circumstances may have had less impact since workers had a greater chance to have a consistent therapy by a psychologist. Thus, use of an audit of medical files may have caused some bias in measuring guideline adherence.

Return to work and physician's management of mental problems
A strength of this study is that a new set of PI has been developed and rated in a new study population by independent researchers. In addition to ten existing and validated PI (9,13,17), this extended set of 20 PI has been systematically developed and validated to cover the whole treatment process, incorporating recent findings. This may have resulted in a lack of variability of the performance rates as some indicators appeared to be non-applicable for many patients. Although performance rates in this study were based on a profound, consistent, and democratic way of scoring by a multidisciplinary team of three researchers (OP, psychologist, and health scientist), we must conclude that validity of the 20 PI remains insecure.
The OP lack of guideline adherence in this study might be attributed to insufficient training in treatment according to the guideline or a possible inapplicability of the trained skills. Multifaceted interventions have proven to be most effective in implementation of evidence-based guidelines (32-35) The 3-day training course on the use of the guideline might have been too minimal for the OP to actually learn the necessary skills. However, Smits et al (16) showed that a comparable course in the guideline for OP-in-training resulted in improved guideline adherence. The same training course resulted in an effective intervention in an earlier trial, but this was in an in-company setting and with a population of workers with adjustment disorders (22).
Optimal implementation of the guideline in occupational healthcare seems unrealistic. The changing work environment in the last decades has created a difficult position for OP, as the varying interests of patients, employers, and the management of occupational health services put pressure on the OP. The training's lack of effect on guideline adherence compared to usual care may be due to these work conditions and perceived behavioral control (19,30). The guideline promotes an intensive form of treatment, for instance in prescribing regular consultation with the general practitioner, which is not applicable in current Dutch practice (3,18,20). Although the (training in the) guideline considers the time constraints under which OP work, OP may have been just too busy to carry out the full intervention.
Recently, the OP guideline has been revised (36). Future research should investigate how guideline adherence can be improved (37,38). Answering the question of why guideline adherence is low and acting on this answer could be a greater influence on the effectiveness of the guideline than changing its contents alone (39). Additionally, measurement of guideline adherence needs to be further developed and complete reporting of information relevant to process evaluation should be enforced and facilitated (40).

Concluding remarks
In this study, we assessed OP adherence to the guideline for the management of workers with common mental health problems and compared this to usual care. Guideline adherence was 50% in both the intervention and control group but appeared to predict return to work especially regarding (new) indicators of progress evaluation. These results indicate that the guideline should focus on these elements to shorten the work absence of workers with common mental health problems.