Enhancing evidence-based advice of occupational health physicians

evidence-based advice Objective This study attempted to determine the effectiveness of an intervention to enhance evidence-based advice given by occupational physicians. Methods Altogether 106 occupational physicians were cluster randomized into 16 groups. The intervention group received a course in evidence-based medicine, followed by a 4-month period of case–method learning sessions in peer groups once every 2 weeks. During these sessions, the participants discussed their patients with respect to sickness absence and the existing evidence for return-to-work prognosis and effective interventions. The participants were assigned to perform a literature search at least once every 4 weeks. The primary outcome measure was the quality of advice based on the correct assessment of prognosis for return to work or the correct choice of return-to-work interventions. Secondary outcome measures were the quality of the searches performed by the intervention group, the use of evidence by all occupational physicians during the intervention period, and the potential predictors for advice quality. Results Better return-to-work interventions were advised by the intervention group than by the control group after 2 months (88% versus 67%, P=0.01), but the difference had decreased after 4 months (76% versus 62%, not significant). No better assessments of prognosis were found for return to work. Most of the searches had a good quality (83% or 73%), and good searching was a positive predictor for a good choice of advised interventions (P=0.03). Without obligatory searches, no increase in evidence use was found. Conclusion Evidence-based advice by occupational physicians in sickness absence episodes can be improved with multifaceted intervention. The actual search for evidence is an essential element.

Employees or employers in need of advice from their occupational health physician expect optimal occupational health care and a high standard of professional performance (1). To ensure such a high standard, the use of scientific evidence in the decision-making process of occupational health physicians has been advocated by researchers in the occupational health field and by international bodies such as the International Commission for Occupational Health (ICOH) and the World Health Organization (WHO) (2)(3)(4). However, several studies in occupational health have shown that there is ample room for improvement in the quality of care, and they give clear indications that the use of available evidence in the scientific literature or evidence-based guidelines could be an important vehicle for this purpose (5)(6)(7)(8). While the application of evidence in the decision-making process of occupational health physicians is still not routine in daily practice, the potential impact of the use of evidence in the decision-making process of clinical physicians has already been illustrated by several studies showing up to 60% improved decisions if evidence was searched for or provided (9)(10)(11). Therefore, in this study, we strived to stimulate the implementation of evidencebased medicine in occupational health to enhance further the professional performance of occupational health physicians and improve occupational health care.
Recently, a promising intervention study showed that teaching skills in evidence-based medicine can actually change physicians' behavior to use more evidence-based therapies if a multicomponent intervention employing evidence-based medicine is used and evidence-based resources are provided on the hospital network (12). Schaafsma et al In line with this finding, we developed and evaluated a multifaceted intervention involving a combination of an educational intervention with case-method learning sessions and recurrent peer meetings. During these meetings, cases of daily occupational health practice were discussed. To stimulate evidence-based-medicine practice further, the occupational health physicians were obliged to search regularly for evidence related to their case. We expected this multifaceted intervention to overcome barriers to the adoption of new behavior in the occupational health setting (13).
Within the Netherlands, a substantial part of daily routine in occupational health practice is involvement in advising patients and their supervisors with respect to return to work. For this purpose occupational health physicians need to have adequate knowledge of the expected duration of sickness absence in relation to work circumstances, and they need to advise employees or their supervisors in such a way that optimal rehabilitation or return to work is realized. In this study, good advice was based on either the correct assessment of prognosis for return to work or the correct choice of return-to-work interventions by an occupational health physician. A correct choice of return-to-work interventions was expressed as the optimal choice of all possible actions or advice the occupational health physician can give to the employee or his supervisor to optimize the patient's healthy return to work.
The purpose of this study was to answer the following questions: (i) does the intervention improve the quality of the occupational health physicians' assessment of return-to-work prognosis and the correct choice of return-to-work intervention for sick-listed employees according to the literature and an expert consensus when compared with the quality of the assessment of a control group, (ii) what is the quality of the searches for evidence performed by the occupational health physicians and can answers be found, (iii) does the intervention enhance the use of evidence also without obligatory literature searches, and (iv) are the characteristics of occupational health physicians, their scores in knowledge and skills in evidence-based medicine, or search qualities predictors of better advice quality?

Study population
We recruited registered occupational health physicians within occupational health services or private practices in the Netherlands via written invitations and information sessions about the study. A necessary sample size was not calculated, but beforehand we decided on a maximum of 100 participants for practical reasons because of the intensity of the planned intervention. Occupational health physicians from the same occupational health service or geographically closely situated services and practices were clustered in peer groups of 6-10 members. An independent research assistant provided a computer-generated blocked randomization sequence with randomization on the group level. All of the occupational health physicians who agreed to participate in the study and who completed the baseline questionnaire entered the trial according to the treatment group their group was assigned to. The participants were not blinded to the treatment assignment. However, the researchers who scored the study outcomes were blinded to the treatment allocation at all times.

Intervention
First, the occupational health physicians in the intervention group were trained in evidence-based medicine for 1.5 days during a period of 2 weeks. In this course, the participants learned the basic steps of evidencebased medicine (eg, occupational health physicians were stimulated to start with a search for relevant practice guidelines and, if necessary, to continue searching for an answer in PubMed). Second, during the follow-up period, the occupational health physicians received several stimuli on the application of evidence-based medicine (eg, newsletters, articles on evidence-based medicine, and several search strategies for PubMed). Third, they were facilitated by access to full text articles, and they could contact a helpdesk for questions on searching the literature. Fourth, obligatory case-method learning sessions took place every 2 weeks during a period of 4 months within the peer groups as organized before the randomization. During these sessions, the physician's own cases of sick-listed employees from daily practice were discussed in a prestructured way with an emphasis on available evidence for all occupational health aspects, following the instructions of Sackett et al for presenting a patient at follow-up rounds (14). The first two sessions and the fifth session were facilitated by one of the authors. Fifth, at least once in every 4 weeks, the occupational health physicians were obliged to perform a literature search for evidence with regard to one of their own cases.
The control group also had access to full-text articles during the intervention period but, in contrast to the intervention group, the participants were not actively stimulated to make use of this facility. After the intervention period, the control group also received the theoretical evidence-based-medicine course of 1.5 days.

Measurements
The baseline questions included demographics of the participants (age, gender), medical experience, experience as an occupational health physician, information on previous training in evidence-based medicine, and experience with research or critical appraisal. The participants of the intervention group were asked to send in (every 2 weeks) the case files used at the case-method learning sessions. The participants of the control group were also asked to send in two case files at the beginning, after 2 months, and after 4 months. All of the case files were documented on a prestructured form with a clear description of the case, the assessed return-towork prognosis, and the intervention advised by the occupational health physician. An independent research assistant selected one case file per occupational health physician to be was sent at the start of the intervention period (T 0 ), after 2 months (T 1 ), and after 4 months (T 2 ) by both groups (see figure 1). The case files had to be filled out completely to be considered usable. In addition, the participants of the intervention group were asked to send us documentation on their performed literature searches. These searches were documented on a prestructured form with a clear description of the search question, the used search strategy, the critical appraisal of the literature, and the answer to the search question.
[See appendix 1.] On the basis of the diagnosis or problem orientation of the selected case files, a panel of three authors with expertise in evidence-based medicine and occupational health (FS, CH, and PS) searched for evidence for return-to-work prognosis and the preferred choice of return-to-work intervention. The evidence retrieved was divided into four level grades, of which grade 1 was considered the highest. [See appendix 2.] From both groups, all of the background details of the case files from the sick-listed employee were noted (ie, age, gender, job, period of sick leave, and diagnosis). Second, the level of evidence referred to by the occupational health physician on return-to-work prognosis or intervention was compared with the level of evidence retrieved by the panel of experts in evidence-based medicine. Third, all of the case forms from both groups were blinded for the group assignment of the occupational health physicians, appraised by all three authors for the correct assessment of prognosis (yes, no) and choice of intervention (yes, no) advised by the occupational health physician. The final evaluation of correctness was decided in consensus. Fourth, all searches at T 1 and T 2 of the intervention group were scored for the type of question searched (diagnosis, etiology, prognosis, therapy), the quality of the search (appendix 1), and possible adjustment to the original prognosis or choice of intervention. Finally, the after-search prognosis or intervention advice by the intervention group was again appraised by all three authors for its correctness (yes, no).

Statistical analysis
The data of all of the participants who entered the trial were analyzed. Differences in the baseline characteristics were tested with t-tests or the Mann Whitney U-test for continuous variables that are either normally or nonnormally distributed, and chi-square tests were used for categorical variables. First, to analyze whether the (T 0 = from the two case files sent in by both groups, one case file was selected at random by an independent research assistant; T 1 = from the two case files sent in by the control group, one case file was selected at random by an independent research assistant; the case file and relevant search log from the intervention group that was sent in after 2 months was selected by an independent research assistant; T 2 = from the two case files sent in by the control group, one case file was selected at random by an independent research assistant; the case file and relevant search log from the intervention group that was sent in after 4 months was selected by an independent research assistant, and O = EMB course, x = case method learning session, ↑ = measurement)

Figure 1. Time frame of the intervention study
Intervention group (N=48)

T0 :
From the two case-files sent in by both groups, one case-file was selected at random by an independent research assistant. T1: From the two case-files sent in by the control group, one case-file was selected at random by an independent research assistant. The case-file and belonging search log from the intervention group that was sent in after two months was selected by an independent research assistant.

Schaafsma et al
intervention had an overall effect over time (intervention × time) for the correct assessment of prognosis and correct choice of intervention, a mixed model analysis based on repeated measurements with adjustments for cluster randomization was performed comparing the intervention group with the control group. The mixedmodel analysis was performed with the SAS procedure GLIMMIX (SAS Inc, Cary, NC, USA), and we chose for the intercept and the clusters to be random and the time points (T 0 , T 1 , T 2 ) to be repeated. We analyzed the differences between the two groups on both the original and after-search correct assessment of prognosis and the choice of intervention for the same patient case to measure the additional effect of the obligatory search. If a statistically significant overall effect was found, posthoc tests were performed for each time point using t-tests within GLIMMIX. We described the type and quality of the searches performed. Second, we analyzed the level of evidence referred to by all of the participants on their case files using the mixed-model analysis for repeated measurements to compare the two groups. Third, separate analyses were performed within the intervention group to investigate potential predictors for the correct assessment of return-to-work prognosis and the correct choice of return-to-work intervention. We performed a subgroup analysis on the effect of gender, experience as an occupational health physician (median of ≥12.5 years), and experience in the past with evidence-based medicine, research, or critical appraisal. We used the scores of the intervention group from our previous study on knowledge and skills in evidence-based medicine after the evidence-based-medicine course to analyze differences in the quality of the search and the quality of advice over time within the intervention group (Schaafsma F, Hugenholtz N, de Boer A, Hulshof, C, and van Dijk F. Enhancing Knowledge and Skills, and Behavior Towards Evidence-based Medicine in a Nonclinical Setting: a Cluster Randomized Controlled Trial, submitted for publication). Statistical analyses were performed using SAS 9.1 and SPSS 11.0 (SPSS Inc, Chicago (IL), USA). P-values of <0.05 were considered statistically significant.

Results
Sixteen occupational health services and eight private practices supplied eligible occupational health physicians. Overall, 131 occupational health physicians were recruited for the trial between May 2005 and September 2005. However, 25 occupational health physicians, altogether from both groups, withdrew before we started the actual intervention, mainly due to the time-consuming elements of the intervention. At T 0 we selected 48 case files from the intervention group and 58 case files from the control group. After 2 months (T 1 ), we selected 41 usable case files from the intervention group, and 49 usable case files from the control group. After 4 months (T 2 ) we selected 45 and 53 usable case files from the intervention and control groups, respectively. [See figure 2.] Table 1 shows the personal characteristics of the 106 participating occupational health physicians at baseline. The participants in the intervention group were older (48 versus 45 years) and had more clinical experience (20 versus 17 years) than those in the control group. No differences in previous experience with evidence-based medicine or other baseline measurements were found. Table 2 shows the characteristics of the sick-listed employees described in the selected case files of all the occupational health physicians. The employees selected for the cases by the intervention group had a longer sick-leave duration at the time of the presentation (21 versus 15 weeks, nonsignificant) and had more variety in disorders than the employees selected by the control group (chi-square P<0.05).

Process variables
All of the occupational health physicians randomized to the intervention group received the evidence-based medicine course at the beginning of the intervention. On the average, each occupational health physician attended 7.2 case-method peer group sessions and turned in an equivalent number of cases. Consequently, on the average, 3.3 literature searches (half of the number of cases) were performed by every occupational health physician. Ninety-one full text articles were requested by 27 occupational health physicians in the intervention group. Five full text articles were requested by three occupational health physicians in the control group. Table 3 shows the evaluation of the assessment of the return-to-work prognosis and the proposed intervention based on the diagnosis of case files before the search for evidence for both groups and after the search for the intervention group. Before the search assignment, we found no statistically significant differences over time between the two groups with respect to a correct assessment of the prognosis (F-test 0.15, df=179, P=0.86) or the choice of intervention (F-test 1.05, df=184, P=0.35). After the search assignment, we found a statistically significant difference over time between the two groups for the correct choice of intervention (F-test 4.54, df=184, P=0.01), but not for the correct prognosis assessment (Ftest 0.47, df=179, P=0.63). The posthoc tests showed an enhancement of 17% at T 1 and 18% at T 2 with respect to good intervention advice for the intervention group. This situation resulted in a statistically significant difference in the advice on good return-to-work intervention with the control group at T 1 (88% versus 67%, T-test -2.22, df=184, P=0.03), and no statistically significant difference at T 2 (76% versus 62%, T-test -1.36, df=184, P=0.17). When all of the adjustments for the cases of the intervention group were combined (assessment of return-to-work prognosis and intervention advice), we  found an improvement in 10 out of 41 cases at T 1 (24%) and in 9 out of 45 cases at T 2 (20%).

Availability and use of evidence
Before the search assignments, we found no statistically significant differences over time between the control and intervention groups with respect to the level of evidence on which the occupational health physicians based their  Table 4. Type and quality of searches performed by the intervention group at T 1 (N=41) and T 2 (N=45).  Table 3. Evaluation of the assessment of the prognosis and choice of intervention for return to work before the obligatory literature search for the intervention and control groups and after the search for the same case for the intervention group at T 0 (N=48, N=58), T 1 (N=41, N=49) and T 2 (N=45, N=53).  Characteristic of the employee in the case file assessment of prognosis or choice of intervention (results not shown). Most of the occupational health physicians in both groups (prognosis 63%, intervention 64%) did not use any evidence or evidence-based sources for their cases. The expert panel found a high level of evidence for most of the selected case files. [See appendix 2.] For the return-to-work prognosis, grade-1 evidence was found for 57% of the case files on the basis of the diagnosis mentioned by the occupational health physician, while grade-2 evidence was found for 19%, grade-3 evidence for 11%, and no evidence for 13%. For the return-towork interventions, the expert panel found grade-1 evidence for 53% of the case files, grade-2 evidence for 14%, grade-3 evidence for 12%, and no evidence for 21%. The provision of high grade-1 evidence was significantly higher for the cases handled in by the control group than for the ones handled in by the intervention group (64% versus 37%, chi-square test P<0.0001). Table 4 shows the literature searches performed by the intervention group. Overall, 86 searches were carried out, mostly on questions about return-to-work interventions or the prognosis at both time points. Examples of the questions are "For a 47-year-old nurse who just had surgery because of a uterus extirpation, how long will it take for her to return to her own regular work?"

Searching for evidence
Or "A technical engineer with 'Morbus Crohn' receives Remicade treatment. Is Remicade treatment more effective in suppressing Crohns' disease than regular treatment with immunodepressants?" However, some occupational health physicians also searched for etiology questions (T 1 N=6, T 2 N=4). An example of an etiology question is "Can bursitis of the elbow be caused by work as a truck driver?"

374
Scand J Work Environ Health 2007, vol 33, no 5 Enhancing evidence-based advice Table 5. Level of evidence recorded by occupational health physicians in the intervention group on their case files before and after an obligatory search for evidence about the prognosis of and intervention for return to work in comparison with the decisions of the experts in evidence-based medicine at T 1 (N=35) and T 2 (N=41). Level  The quality of the searches was considered good by the experts for the greater part of the cases, 83% at T 1 and 73% at T 2 . For most of the questions, an answer was found (T 1 90% and T 2 80%), and, in these cases, original articles found on PubMed were used the most often. In addition to several websites, the guidelines, and textbook used by the occupational health physicians were similar to the sources used by the experts. The Cochrane database was mentioned only twice as a source by the occupational health physicians, while the experts used it seven times. As was to be expected (table 5), we found a statistically significant increase in the level of evidence when the before and after search assignments were compared using the experts as a "gold standard".

Predictors of good advice
Gender, experience as an occupational health physician, experience in the past with evidence-based medicine, research, or critical appraisal were not statistically significantly related over time to the correct return-to-work prognosis or to interventions within the intervention group. Over time, we found that good searches were a positive predictor for a correct choice of return-to-work intervention (F-test, df=184, P=0.03). We did not find a relationship over time between good searches and correct prognosis. We found no relationship between high scores on knowledge and skills measured directly after the evidence-based-medicine course (mean score 124.2, 95% CI 113.7-134.7) and good quality of the searches for evidence for the intervention group (unpublished data: Schaafsma F, Hugenholtz N, de Boer A, Hulshof, C, and van Dijk F. Enhancing Knowledge and Skills, and Behavior Towards Evidence-Based Medicine in a Nonclinical Setting: a Cluster Randomized Controlled Trial.). Nor did we find a relationship between high scores for knowledge and skills and the correct returnto-work prognosis or intervention before or after the search assignment.

Discussion
The multifaceted intervention tested in this trial produced an enhancement of correct return-to-work intervention choice for the intervention group when compared with that of the control group after 2 months, but decreased after 4 months. No enhancement was found for the assessment of return-to-work prognosis. Before the search assignment, we did not find either a significant increase in evidence use by the intervention group or an enhancement of the quality of advice on the case file. Good-quality searches were a positive predictor for a good return-to-work intervention choice.

Strength and limitations
This study was unique in the sense that real cases from daily practice were used to study performance in evidence-based medicine and its potential effect in enhancing quality. The generalizability was therefore high, although its internal validity may have been weakened due to differences in the case files handed in by the two groups (15). The cases of the intervention group were probably more complex, as there was a broader variety in the disorders. The intervention group probably preferred to discuss the cases that raised difficulties or questions in practice during the case-method learning sessions. In addition, the control group handed in more case files for which evidence-based practice guidelines (evidence grade 1) existed as opposed to the intervention group. Within the Netherlands, evidence-based occupational health guidelines are available for several prevalent disorders. The case files from the control group possibly reflect more routine cases and may be regarded as a selection of best practice.
Overall, our estimation of the availability of evidence for these case files could be an overestimation, as all of the case files or patients taken together were the Schaafsma et al denominator in the calculation instead of the separate prognosis assessments or intervention choices, analogous with other studies (16)(17)(18)(19).
The evaluation of return-to-work prognosis and intervention, as well as the evaluation of search quality, was done using an expert panel. The expert panel used and discussed the available evidence on different topics and decided in consensus about the assessment of prognosis and the choice of intervention to overcome the problem of interindividual variability (20). The evaluation of available evidence and search quality by an expert panel is in line with previous experiences on evaluating searches for evidence following the steps of evidence-based medicine and with other similar studies on this topic (16)(17)(18)(19)21). The searches performed by the intervention group concentrated on finding original articles in PubMed. This procedure could be expected because the information given during the evidencebased-medicine course focused on searching in PubMed. However, the instruction on the search file suggested to start with a search for relevant guidelines, which is in line with the latest insight that practitioners should be stimulated to become more users and not so much doers of evidence-based medicine (22). For them, it is not always necessary to search and appraise original articles themselves. If a guideline or other evidence-based summary exists on a certain topic, it is sufficient to use only this source. It is possible that these occupational health physicians were not sufficiently made aware of all evidence-based information sources, as is shown by the limited use of the Cochrane database as opposed to its use by the experts. Although searches for return-towork prognosis and interventions were equally divided, we especially found an enhancement of the correct choice of return-to-work interventions predicted by good searching. This finding may be explained by the fact that some of the occupational health physicians who had searched for prognosis or etiology questions still changed their choice of intervention instead of, for example, their originally assessed prognosis. Besides, assessment of return-to-work prognoses may be influenced more strongly by contextual factors of the specific case than a choice of best intervention is. Retrieving "new" evidence may then not lead so much to adjustments of the original assessment.
The expected positive influence of the evidencebased-medicine course and peer sessions did not, by itself, persuade the participants to search and use more evidence in their advice towards their employees on sick leave. The exchange of tacit knowledge and explicit knowledge during the sessions apparently was not yet translated into better advice quality. Maybe the participant's knowledge that they had to perform a search for evidence every month anyway prevented them from spontaneously relying on evidence for their advice.

Comparison with other studies
This study is the first trial in occupational health to test the effect of evidence-based medicine implementation in improving professional quality in daily practice. Within other medical disciplines randomized controlled trials have been performed with good results in changing professional behavior into more evidence-based practice. However, most of these randomized controlled trials tested evidence-based medicine behavior via selfassessment, while only a few have studied the effect of evidence-based medicine practice on a specific outcome variable (23,24). Several cross-sectional studies have been performed within various medical disciplines on the extent to which daily practice was based on sound evidence. These studies searched evidence for the diagnosis intervention combination within hospital wards or practices of physicians during a limited period of time (16)(17)(18)(19). In our study, the experts not only searched for best evidence on diagnosis-return-to-work prognosis and diagnosis-return-to-work intervention combinations, but also compared the results with the evidence used by the occupational health physician. In addition, we evaluated progress after the obligatory searches. One of the main findings of our intervention, the adaptation of the original choice for return-to-work intervention due to retrieved evidence, is in line with findings in clinical settings. For example, in oncology, studies have shown that, if evidence had been provided when needed, the decisions could have been different in 30-60% of the times (9).
In our previous study on the competence of evidence-based medicine, we concluded that case-method learning sessions with peers in combination with a theoretical course in evidence-based medicine and continuous positive stimulants to practice evidence-based medicine enhances knowledge, skills, and behavior in evidence-based medicine in nonclinical settings (Schaafsma F, Hugenholtz N, de Boer A, Hulshof, C, and van Dijk F. Enhancing Knowledge and Skills, and Behavior Towards Evidence-based Medicine in a Nonclinical Setting: a Cluster Randomized Controlled Trial; unpublished observations). However, the results of our present study have shown that the predictive value of competence in evidence-based medicine is low with regard to actual performance in evidence-based medicine, as has been shown by others (25). An improvement in the quality of professional advice needs a concrete search for evidence, as only then will professional quality be enhanced in daily practice.

Policy implications and indications for further research
To ensure high professional quality for occupational health care, the actual practice of evidence-based medicine is essential. Therefore, frequent searching of relevant literature and databases and applying the findings in daily practice is necessary. This study has shown that a substantial amount of evidence is available for the use of occupational health physicians in their daily practice when they must offer advice on sickness absence. For most of the cases in this study, a good search resulted in an answer that could enhance the quality of the advice of the professional. The extent to which the case-method learning sessions with peers stimulated evidence-basedmedicine practice needs further research. Occupational health physicians and the management of occupational health services should invest in learning evidence-basedmedicine practice and feel obliged to do real searches of good quality. For this purpose, repeated sessions with PubMed tutorials can be helpful, and more information on available evidence-based sources and support by a good knowledge infrastructure is essential (26).

Concluding remarks
Professional performance of occupational health physicians can be improved with multifaceted intervention combining a course in evidence-based medicine with repeated case-method learning sessions with peers and regular search assignments. The actual search for evidence proved to be an essential element in enhancing the quality of advice given by occupational health physicians with respect to sickness absences.