Study population

Data were from one of the participating hospital districts of the Finnish Public Sector study, which is a prospective cohort study of public sector employees from 10 municipalities and 21 hospitals in the same geographical areas in Finland (28). The subset of shift workers was chosen because study participants had data from both a questionnaire survey on predictors of work disability and administrative data on working hours and workplace characteristics. We included participants who responded to the survey in 2015, were linked to administrative data in the same year, and had a minimum follow-up period of 150 days for sickness absence after 2015. All participants were eligible for sickness absence. Approval was obtained from the ethics committee of the Helsinki-Uusimaa Hospital District Ethics Committee (HUS/1210/2016).

Potential predictors of long-term sickness absence

Participant-completed questionnaire survey items of the validated FIOH risk calculator for the prediction of work disability in the general working population include: age group (<35; 35–39; 40–44; 45–49; 50–54; ≥55 years), body mass index (BMI) (<18.5, 18.5–<25; 25–<30; ≥30 kg/m²), socioeconomic status (SES) (low; intermediate; high), smoking (yes or no), number of chronic diseases (0; 1; 2; ≥3), self-rated health (good; fairly good; moderate; fairly poor; poor), difficulty falling asleep (not at all; 1–3 times/month; about once a week; 2–4 nights/week; 5–6 nights per week; every night), and the number of sickness absence episodes >9 days during one year before baseline (0; 1; 2; ≥3) (26).

Administrative records comprised individual-level data on 18 characteristics of working hours (27). The times of daily working hours and the reasons for an absence (day off, sick leave, maternity leave, physician’s on-call duties, annual leave etc) were retrieved from a shift scheduling software Titania® (CGI Finland) and annual working hours were calculated. The variables from these records were calculated for those who had ≥150 working days in 2015 and are presented in detail in table 1. Further administrative records for the year 2015 were retrieved from human resources management registries of the workplaces. Using these records, we defined ten characteristics of the participant’s working unit: the number of employees, mean age of employees, proportion of employees aged ≤30, proportion of employees aged ≥60, proportion of non-permanent employees, proportion of nurses, proportion of employees with low SES, turnover (last 2 years; between autumn 2013 and autumn 2015), turnover (last 4 years; between autumn 2011 and autumn 2015). Low SES referred to ‘service workers’, ‘process workers’ and ‘other/elementary occupations’, according to the International Standard Classification of Occupations (29). In line with data protection regulations, we excluded data from small work units (N<5) from the analysis and computed these characteristics only for work units comprising ≥5 employees (figure 1).

A complete list and categorization of the potential predictors is provided in table 1 and the supplementary material,www.sjweh.fi/article/4124, table S1.

Ascertainment of long-term sickness absence during the follow-up

In Finland, sickness absence periods lasting >9 days and with medical certification are recorded in the Social Insurance Institution. Employees receive compensation based on their salary during their sickness absence for up to 300 weekdays. If a sickness absence lasts ≥90 days, to be entitled for compensation, the employee needs to provide the Finnish Social Insurance Institution with a detailed certificate from an occupational physician about his/her inability to work (30).

We used administrative data to ascertain sickness absences at follow-up, including long-term sickness absence records between 1 January 2016 and 31 December 2019. As in previous studies, outcome was the first ≥90-day sickness absence (14). The first >30-day sickness absence was our secondary outcome.

Statistical analyses

There were no missing data on administrative records at baseline or sickness absence at follow-up. The amount of missing values for the 8 questionnaire items was small: 3% for BMI (N=101) and the number of chronic diseases (N=92), <1% for smoking (N=18), self-rated health (N=12) and trouble falling asleep (N=25). As these questionnaire items were categorical, we imputed missing values using the mode value from the entire study population. For comparison, we repeated the main analyses in the 2981 participants with no missing data (93.2% of the total sample, the complete case analysis).

Analysis of long-term sickness absence prediction included the following three steps. First, we examined separately three predictive models including (i) all 8 pre-defined questionnaire-based items (ii), all 28 administrative data-based items, and (iii) all 36 items from both sources. We obtained item coefficients from logistic regression and constructed a risk score for each participant using the predicted risk from the model. We compared the three prediction models to determine which data source or sources provided the best prediction for long-term sickness absence as indicated by C-index and other prediction metrics. C-index gives the probability that a randomly selected individual who experienced the outcome during the follow-up, had a higher risk score than a randomly selected individual who did not experience the outcome. The C-index ranges from 0.5 (no predictive ability) to 1 (maximum predictive ability). C-index under 0.7 represents poor, 0.7–0.8 good, and >0.8 strong discrimination ability (31). As recently suggested (32, 33), we additionally examined detection rate, false positive rate, and the ratio of true to false positives using different thresholds for stratifying the population into high risk (test positive) versus low risk (test negative). Formulas for these statistics are as follows:

False positive rate (the proportion of test positive cases who did not experience work disability) = b/(b+d)

Detection rate (the proportion of work disability cases who were test positive) = a/(a+c)

Ratio of true to false positives = 1: (b/a), where a, b, c and d represent different combinations of risk scores and work disability as defined below:

The alternative thresholds for a test positive result were set at 5%, 10%, 15% and 20% of model predicted risk.

To evaluate whether the best model was associated with the outcome at each year of the follow-up, we computed cumulative incidence of long-term sickness absence by follow-up year (1–4) and high versus low risk score. Relative importance of each predictor was illustrated using -10log(P-value).

Second, to examine the robustness of evidence, we investigated whether the same source or sources as in the main analysis provided the best prediction also after reducing the number of predictors. For dimension reduction (feature selection), we used several alternative approaches, including traditional stepwise logistic regression modelling and five different methods of machine learning with cross-validation. We used stepwise logistic regression with forward and backward variable selection based on Akaike Information Criteria (AIC). Machine learning models included: (i) LASSO (least absolute shrinkage and selection operator) regression, which is a regression analysis method that performs feature selection and regularization. This method performs L1 regularization. The results were based on k-fold cross-validation with k=10 (34); (ii) ridge regression, which is a model tuning method that is used to analyze any data that suffers from multicollinearity. This method uses L2 regularization. The results were based on k-fold cross-validation with k=10 (34); (iii) elastic net, which is a regularized regression method that linearly combines the L1 and L2 penalties of the Lasso and Ridge methods – the results were based on repeated k-fold cross-validation with k=10 (34); (iv) genetic algorithm (GA), which is a stochastic search algorithm inspired by the basic principles of biological evolution and natural selection – the results were based on k-fold cross-validation with k=10 (35); and (v) Boruta, which is a random forest-based feature selection algorithm for finding relevant variables (36).

Instead of generating new risk scores, the objective of these analyses was to determine whether variable selection would favor questionnaire-based or register-based predictors or encompass predictors from both data sources.

Third, we repeated the main analysis as described in step 1 using the first >30-day sickness absence (instead of the first >90-day sickness absence) as the outcome.

All analyses were performed using SAS 9.4 (SAS Institute, Cary, NC, USA) and R 4.2.2, including packages Boruta (36), glmnet (34), caret (37), and odds.n.ends (38).

Comparison with other studies

Several studies on sickness absence prediction have been conducted, with risk prediction models typically showing varying predictive capacities. C-indices or AUC for a prediction model for sickness absence have been as high as 0.80 (16), while some studies have presented models with good discrimination (C-index or AUC less than 0.80 but exceeding 0.7) (7, 10, 14) or modest discrimination values (C-index or AUC<0.7) (11, 13). However, we are not aware of previous studies comparing the predictive performance between models using different combinations of questionnaire and administrative data. Furthermore, few studies have reported detection and false positive rates and the ratio true-to-false positives of the models, although these metrics are important to assess the clinical value of prediction algorithms.

In recent years, an array of clinical tools for predicting the risk of specific chronic diseases have been established, some of which are already in use in clinical practice. The predictive capacities of these models are comparable with those for long-term sickness absence. For example, the Pooled Cohort Risk Equations for predicting the 10-year risk of cardiovascular disease events yielded a C-index of 0.72 (39), whereas the C-index of the FINDRISK model for predicting type 2 diabetes have varied between 0.74 and 0.77 (40, 41), and that for the QRISK3 model for the prediction of cardiovascular disease risk between 0.70 and 0.91 (42).

We have previously validated a risk score for work ability, and further tested its predictive performance among employees with chronic conditions (26, 43). Among the different disease groups, with an estimated 30% risk cut-off, the detection rates varied between 42% and 80%, and false positive rates between 10% and 46%, depending on the disease group under investigation. For a lower cut-off (5%) for test positive result, the ranges of the corresponding indicators were 92–99% and 54–94%, respectively.

One previous study of prediction models for workplace bullying was conducted using both survey responses and administrative records (44). The predictive performance of the models obtained was modest, with C-indices <0.7. For a low (5%) cut-off for a positive test result of being bullied, the authors found relatively high detection rates (79–84%) although this was accompanied by high false positive rates (56–71%). With a 15% cut-off, the false positive rates were low (1–5%), but the detection rates dropped to 3–17% (44). These results are comparable to those from our study, in which a low cut-off point (5%) yielded a detection rate of 75%, whereas a 20% cut-off point reduced the detection rate to 27%. The corresponding false positive rates were 33.8% and 3.5%, respectively.

Strengths and limitations of this study

Strengths of this study include the use of informative metrics to evaluate the clinical value of the prediction algorithms, the assessment of predictors across two sources of data and ascertainment of long-term sickness absence through administrative records covering all employees. In addition to C-statistics, we evaluated clinical value with detection and false positive rates and the ratio true to false positives. Few previous studies on prediction models for sickness absence have reported these metrics (45).

There are, however, some important limitations. No independent study population was available for external validation or examination of the reproducibility of our findings. As long-term sickness absence is defined as temporary work disability, receipt of a sickness absence is associated with medical and non-medical factors, such as sickness absence regulations, the work environment, the nature of the job, and the extent to which a workplace is prepared to accommodate the disability. In addition, our target population was limited to shift-working hospital employees and included more women than men. Thus, the findings of this study may not be generalizable to other settings and should be validated in other countries and study populations.

Concluding remarks

In conclusion, when evaluating the utility of alternative prediction algorithms for long-term sickness absence among shift workers, we compared models based on employee questionnaires and administrative records. Our findings revealed that neither source outperformed the other; instead, the highest predictive accuracy was achieved when both data sources were combined.

In practice, combining responses from two different sources may not always be straightforward to implement because occupational health services do not usually have access to administrative data and the employers mainly do not have the authority to handle sensitive self-report data such as chronic conditions and self-rated health needed for survey responses. According to our results, both sources alone performed reasonably well separately and could therefore be recommended to be used for risk prediction.

Our findings can guide the implementation of targeted interventions, as decisions concerning target groups often necessitate the use of dichotomized risk models to identify individuals at risk – in this context, those with a high probability of experiencing long-term sickness absence. For meaningful resource allocation, the performance of the prediction models is important, and the intervention’s characteristics should be factored into the threshold chosen for a positive test result. When implementing cost-effective interventions with no negative side effects, it is preferable to set low risk thresholds for a positive test result. This approach enhances detection rates while simultaneously accommodating relatively high false-positive rates. However, in cases where the intervention is costly or has an unknown safety profile with potential adverse effects, minimizing false-positive rates is favored to mitigate harms. In the present study, models allowing a correct prediction for the majority of the study population had relatively high false positive rates suggesting that these predictive algorithms are more suitable for the implementation of inexpensive interventions with no negative side effects.

Ethics approval and consent to participate

Ethical approval was obtained from the ethics committee of the Helsinki-Uusimaa Hospital District Ethics Committee (HUS/1210/2016). Informed consent was obtained from all individual participants included in the study.