Commentary

Scand J Work Environ Health 2015;41(1):102-104    pdf

doi:10.5271/sjweh.3457

Who is the expert for the evaluation of work ability?

by Torén K, Järvholm B

The decision to award a disability pension includes a formal procedure that varies between countries. Medical doctors are often used as experts in this process, sometimes they also make the final decision. The study by Dell-Kuster et al (1) shows how evaluations can differ among physicians. The judgments between the physicians who treat the patients and a group of specialized physicians differed considerably. The differences reported in the study is probably greater than in a random sample of patients who apply for disability pension as the study includes cases that have been referred for further investigation by a disability insurance officer due to inconsistencies in the assessments. However, the differences raise a question about the validity of judgments and justice – do similar cases have the same evaluation?

The criteria for disability pension varies between countries, but it includes an evaluation of the disease and the functional impairments caused by the disease, and also the possibilities to work and get a job despite the impairments. Work is a not a clearly defined subject. You can work in the garden (ie, take care of your roses) but be unable to perform most paid work if your “productivity” is too low for employers’ needs. Many people with diseases work. A Finnish study of public employees around 60 years of age found that 74% had at least one chronic disease (2). Also workers with decreased productivity may work. An average decrease of 34% for productivity was reported among Finnish workers with upper-extremity disorders who were not on sick leave (3). A Dutch study found a considerable loss of productivity a year after workers had returned to work after having being on sick leave for musculoskeletal disorders (4).

The possibility to work while having a disease depends on the individual’s possibility to adjust his work. Geuskens et al (5) followed persons with newly onset arthritis during 12 months and found that 39% adapted the work, mainly through changing the work environment or working time. They also found that clinical factors and reduced coping had no influence on sick leave when pain and function were taken into account. Both coping strategies and external factors (eg, lack of support from colleagues and supervisors) have been reported as barriers to or facilitators of working with arthritis (6–8). Thus, a claim for disability pension due to disease is not just dependent on the severity of the disease(s), but also the employer’s willingness to adjust the work and perhaps accept a lower level of productivity along with the willingness of colleagues to adapt accordingly.

Several studies have indicated determinants for a person to have disability pension, eg, showing that low socioeconomic class, poor health, obesity, and smoking habits influence the risk. However, there are rather few studies on those who assess disability pension. The inter-rater agreement was reasonable (average 76%) when Dutch insurance physicians assessed the physical and mental functional capacity of written structured evaluations from trained nurses, while the inter-rater agreement was poor when physicians assessed the number of hours the claimant could function daily (9). A previous Dutch study –where four insurance physicians all examined and evaluated the same 30 patients with low-back pain – also showed poor inter-rater agreement (10). Another Dutch study of insurance physicians found modest inter-rater reliability in assessing patients with depressive disorders (11).

A rather high inter-rater variability is common among medical specialists, eg, there is considerable variability among radiologists interpreting mammographies or describing lesions in sonography and pathologists describing dysplasia in the gastrointestinal system (12, 13). Among physicians (GP, internists and cardiologists) who rated the risk for coronary event and the need for lipid-lowering treatment in the same nine cases (14), there were considerable differences in rating risk and need for therapy even among physicians of the same specialty (eg, the concordance in risk estimates in high risk cases was only 0.31 among cardiologists). Furthermore, many more internists than GP wanted to start therapy in low-risk cases (45 versus 27%). High inter-rater variability may also occur in asthma diagnosis. In a study from Gothenburg, Sweden, 86 patients with asthma diagnosis from two primary healthcare units were re-evaluated by a specialist in allergic diseases. Asthma was verified in 57 patients (66%) (15). Fourteen pediatric asthma specialists were asked to classify eight patients classified as having asthma or asthma-symptoms (16).The agreement for diagnoses of asthma was very low: the kappa was 0.29. Their agreement was ever lower whether they based their classification symptoms or pulmonary functions tests (kappa=0.19).

Dell-Kuster et al (1) indicated that financial interests may influence the assessments and the insurers paid the Swiss expert evaluators in their study. It is well-known that financial conflicts of interest may influence evaluations. Scientists with links to the pharmaceutical or tobacco industry make evaluations more favorable to the industry than neutral reviewers (17). It would be astonishing if physician evaluations were not influenced by those who pay them.

From the point of the single claimant, the differences in evaluations may seem unacceptable. Medical doctors are typically trained to diagnose and treat patients. However, they have far less training in evaluating the capacities needed in working life. In the present working life, skills such as social capacity, learning new things, coping with stress etc are important for most workers in western societies. The solution is hardly to find the ultimate expert but rather to allow groups of “experts” with different types of expertise to give arguments for and against disability pension. The use of experts with no affiliations with either the insurer or the patient is a way to reduce bias caused by financial interests.

Training experts and making them aware of the variability in evaluations would also be valuable. The use of video presentations for education and training of physicians have been suggested (18). Most patients have more than one diagnosis and several rather unique characteristics making algorithms less useful. Measuring the subjectivity involved in the evaluations and the variability between evaluators will contribute to improve the processes and make the assessors more humble to the sometimes difficult decision of deciding who shall have disability pension.

This article refers to the following texts of the Journal: 2014;40(5):493-501  2009;35(4):301-308  2008;34(6):420-429  2005;31(5):367-374

The decision to award a disability pension includes a formal procedure that varies between countries. Medical doctors are often used as experts in this process, sometimes they also make the final decision. The study by Dell-Kuster et al (1) shows how evaluations can differ among physicians. The judgments between the physicians who treat the patients and a group of specialized physicians differed considerably. The differences reported in the study are probably greater than in a random sample of patients who apply for disability pension as the study includes cases that have been referred for further investigation by a disability insurance officer due to inconsistencies in the assessments. However, the differences raise a question about the validity of judgments and justice – do similar cases have the same evaluation?

The criteria for disability pension vary between countries, but they include an evaluation of the disease and the functional impairments caused by the disease, and also the possibilities to work and get a job despite the impairments. Work is a not a clearly defined subject. You can work in the garden (ie, take care of your roses) but be unable to perform most paid work if your “productivity” is too low for employers’ needs. Many people with diseases work. A Finnish study of public employees around 60 years of age found that 74% had at least one chronic disease (2). Also workers with decreased productivity may work. An average decrease of 34% for productivity was reported among Finnish workers with upper-extremity disorders who were not on sick leave (3). A Dutch study found a considerable loss of productivity a year after workers had returned to work after having being on sick leave for musculoskeletal disorders (4).

The possibility to work while having a disease depends on the individual’s possibility to adjust his work. Geuskens et al (5) followed persons with newly onset arthritis during 12 months and found that 39% adapted the work, mainly through changing the work environment or working time. They also found that clinical factors and reduced coping had no influence on sick leave when pain and function were taken into account. Both coping strategies and external factors (eg, lack of support from colleagues and supervisors) have been reported as barriers to or facilitators of working with arthritis (68). Thus, a claim for disability pension due to disease is not just dependent on the severity of the disease(s), but also the employer’s willingness to adjust the work and perhaps accept a lower level of productivity along with the willingness of colleagues to adapt accordingly.

Several studies have indicated determinants for a person to have disability pension, eg, showing that low socioeconomic class, poor health, obesity, and smoking habits influence the risk. However, there are rather few studies on those who assess disability pension. The inter-rater agreement was reasonable (average 76%) when Dutch insurance physicians assessed the physical and mental functional capacity of written structured evaluations from trained nurses, while the inter-rater agreement was poor when physicians assessed the number of hours the claimant could function daily (9). A previous Dutch study – where four insurance physicians all examined and evaluated the same 30 patients with low-back pain – also showed poor inter-rater agreement (10). Another Dutch study of insurance physicians found modest inter-rater reliability in assessing patients with depressive disorders (11).

A rather high inter-rater variability is common among medical specialists, eg, there is considerable variability among radiologists interpreting mammographies or describing lesions in sonography and pathologists describing dysplasia in the gastrointestinal system (12, 13). Among physicians (GP, internists and cardiologists) who rated the risk for coronary event and the need for lipid-lowering treatment in the same nine cases (14), there were considerable differences in rating risk and need for therapy even among physicians of the same specialty (eg, the concordance in risk estimates in high risk cases was only 0.31 among cardiologists). Furthermore, many more internists than GP wanted to start therapy in low-risk cases (45 versus 27%). High inter-rater variability may also occur in asthma diagnosis. In a study from Gothenburg, Sweden, 86 patients with asthma diagnosis from two primary healthcare units were re-evaluated by a specialist in allergic diseases. Asthma was verified in 57 patients (66%) (15). Fourteen pediatric asthma specialists were asked to classify eight patients classified as having asthma or asthma-symptoms (16). The agreement for diagnoses of asthma was very low: the kappa (κ) was 0.29. Their agreement was ever lower regardless when they based their classification on symptoms or pulmonary functions tests (κ=0.19).

Dell-Kuster et al (1) indicated that financial interests may influence the assessments and the insurers paid the Swiss expert evaluators in their study. It is well-known that financial conflicts of interest may influence evaluations. Scientists with links to the pharmaceutical or tobacco industry make evaluations more favorable to the industry than neutral reviewers (17). It would be astonishing if physician evaluations were not influenced by those who pay them.

From the point of the single claimant, the differences in evaluations may seem unacceptable. Medical doctors are typically trained to diagnose and treat patients. However, they have far less training in evaluating the capacities needed in working life. In the present working life, skills such as social capacity, learning new things, coping with stress etc are important for most workers in western societies. The solution is hardly to find the ultimate expert but rather to allow groups of “experts” with different types of expertise to give arguments for and against disability pension. The use of experts with no affiliations with either the insurer or the patient is a way to reduce bias caused by financial interests.

Training experts and making them aware of the variability in evaluations would also be valuable. The use of video presentations for education and training of physicians have been suggested (18). Most patients have more than one diagnosis and several rather unique characteristics making algorithms less useful. Measuring the subjectivity involved in the evaluations and the variability between evaluators will contribute to improve the processes and make the assessors more humble to the sometimes difficult decision of deciding who shall have disability pension.

References

1 

Dell-Kuster, S, Lauper, S, Koehler, J, Zwimpfer, J, Altermatt, B, Zwimpfer, T, et al. (2014). Assessing work ability - a cross-sectional study of interrater agreement between disability claimants, treating physicians, and medical experts. Scand J Work Environ Health, 40(5), 493-501, http://dx.doi.org/10.5271/sjweh.3440 .

2 

Virtanen, M, Oksanen, T, Batty, GD, Ala-Mursula, L, Salo, P, Elovainio, M, et al. (2014). Extending employment beyond the pensionable age: a cohort study of the influence of chronic diseases, health risk factors, and working conditions. PLoS One, 9(2), e88695, http://dx.doi.org/10.1371/journal.pone.0088695 .

3 

Martimo, KP, Shiri, R, Miranda, H, Ketola, R, Varonen, H, & Viikari-Juntura, E. (2009). Self-reported productivity loss among workers with upper extremity disorders. Scand J Work Environ Health, 35(4), 301-8, http://dx.doi.org/10.5271/sjweh.1333 .

4 

Lötters, F, Meerding, WJ, & Burdorf, A. (2005). Reduced productivity after sickness absence due to musculoskeletal disorders and its relation to health outcomes. Scand J Work Environ Health, 31(5), 367-74, http://dx.doi.org/10.5271/sjweh.920 .

5 

Geuskens, GA, Hazes, JM, Barendregt, PJ, & Burdorf, A. (2008). Predictors of sick leave and reduced productivity at work among persons with early inflammatory joint conditions. Scand J Work Environ Health, 34(6), 420-9, http://dx.doi.org/10.5271/sjweh.1298 .

6 

Allaire, SH. (2004). What work changes do people with arthritis make to preserve employment, and are such changes effective? Arthritis Rheum, 51(6), 871-3, http://dx.doi.org/10.1002/art.20832 .

7 

Lacaille, D, White, MA, Backman, CL, & Gignac, MA. (2007). Problems faced at work due to inflammatory arthritis: new insights gained from understanding patients’ perspective. Arthritis Rheum, 57(7), 1269-79, http://dx.doi.org/10.1002/art.23002 .

8 

Gignac, MA, & Cao, X. (2009). “Should I tell my employer and coworkers I have arthritis?“A longitudinal examination of self-disclosure in the work place. Arthritis Rheum, 61(12), 1753-61, http://dx.doi.org/10.1002/art.24889 .

9 

Spanjer, J, Krol, B, Brouwer, S, & Groothoff, JW. (2008). Inter-rater reliability in disability assessment based on a semi-structured interview report. Disability and rehabilitation, 30(24), 1885-90, http://dx.doi.org/10.1080/09638280701688185 .

10 

Brouwer, S, Dijkstra, P, Gerrits, E, Schellekens, J, Groothoff, J, Geertzen, J, et al. (2002). Intra- en inter-beoordelaarsbetrouwbaarhied ’ FIS-belastbaarheidspatroon’ en ‘Functionele Mogelijkhedenlijst’ [Dutch]. Tijdschrift voor Bedrijfs- en Verzekeringsgeneeskunde, 10, 360-7, cited in 9.

11 

Slebus, FG, Kuijer, PP, Willems, JH, Frings-Dresen, MH, & Sluiter, JK. (2010). Work ability assessment in prolonged depressive illness. Occup Med (Lond), 60(4), 307-9, http://dx.doi.org/10.1093/occmed/kqq079 .

12 

Baker, JA, Kornguth, PJ, Soo, MS, Walsh, R, & Mengoni, P. (1999). Sonography of solid breast lesions: observer variability of lesion description and assessment. AJR American journal of roentgenology, 172(6), 1621-5, http://dx.doi.org/10.2214/ajr.172.6.10350302 .

13 

Eaden, J, Abrams, K, McKay, H, Denley, H, & Mayberry, J. (2001). Inter-observer variation between general and specialist gastrointestinal pathologists when grading dysplasia in ulcerative colitis. The Journal of pathology, 194(2), 152-7, http://dx.doi.org/10.1002/path.876 .

14 

Vancheri, F, Strender, LE, Montgomery, H, Skaner, Y, & Backlund, LG. (2009). Coronary risk estimates and decisions on lipid-lowering treatment in primary prevention: comparison between general practitioners, internists, and cardiologists. European journal of internal medicine, 20(6), 601-6, http://dx.doi.org/10.1016/j.ejim.2009.04.002 .

15 

Marklund, B, Tunsater, A, & Bengtsson, C. (1999). How often is the diagnosis bronchial asthma correct? Family practice, 16(2), 112-6, http://dx.doi.org/10.1093/fampra/16.2.112 .

16 

Baker, KM, Brand, DA, & Hen, J, Jr. (2003). Classifying asthma: disagreement among specialists. Chest, 124(6), 2156-63, http://dx.doi.org/10.1378/chest.124.6.2156 .

17 

Järvholm, B, & Bohlin, I. (2014). Evidence-based evaluation of information: the centrality and limitations of systematic reviews. Scand J Public Health, 42(13 Suppl), 3-10, http://dx.doi.org/10.1177/1403494813516713 .

18 

Van Sickle, D, Magzamen, S, Maenner, MJ, Crane, J, & Corden, TE. (2013). Variability in the labeling of asthma among pediatricians. PLoS One, 8(4), e62398, http://dx.doi.org/10.1371/journal.pone.0062398 .