Recent decades have brought large improvements to health and safety conditions in the construction industry, yet the number of fatal and non-fatal injuries remains extremely high (1). To combat this, some employers have implemented safety incentive programs, such as those that use injury-based safety performance metrics to evaluate overall worksite safety and reward workers. However, these lagging indicator-based programs may discriminate against injured workers (2) and may reduce injury reporting (3, 4).
As an alternative, programs could rely instead on leading indicators of safety, such as hazard control and other root causes of injuries. In partnership with individuals from the local construction industry, we developed a leading indicator-based program, also known as a safety communication and recognition program, B-SAFE (www.northeastern.edu/b-safe) (5). B-SAFE facilitates communication between workers and management regarding hazard controls as identified by safety inspections completed by in-house safety professionals. The program uses frequent (more than once per week) inspections that communicate positive safe working conditions (eg, recognizing the use of hazard controls). The B-SAFE program was designed to be an add-on to an existing health and safety program in which regular safety inspections are part of safety management system. However, the efficacy and effectiveness of the B-SAFE program on changing safety conditions and preventing injuries is unknown.
Our aim was to evaluate the efficacy of B-SAFE on measures of safety at the worksite through a cluster randomized controlled trial (RCT) using a mixed methods approach. We hypothesized that intervention sites would show a greater improvement over time than control sites, in both quantitative and qualitative measures of safety. Quantitatively, our primary outcome was safety climate. Based on the functional construct of organizational climate (6), safety climate is the workers’ perception of what is rewarded and supported at the worksite with regard to safety and its competing messages such as productivity (6, 7). Within the theoretical framework of this definition, we expected the B-SAFE program to improve safety climate as the program should increase communications and provides a reward for all workers when safety working conditions are achieved consistently on a site. In addition, safety climate may act as a proxy for injury outcomes based on its empirical associations of injury measures (7–11) and the theoretical safety performance framework of Neal and Griffin (12, 13). Qualitatively, we explored B-SAFE’s effect on the themes of safety awareness, safety communication, and collaborative competition (themes identified as positively impacted by B-SAFE during our pilot) (5).
Methods
Study design and sample population
We conducted a cluster RCT on four pairs of commercial construction worksites. One pair was recruited from an owner and three pairs from general contractors in the greater Boston area. The clustering was completed at the general contractor/owner pair level as we assumed that the variability between sites of different general contractors/owners would be much larger than the variability of sites within a general contractor/owner. To be included in the study, sites had to be using the online data inspection management program Predictive Solutions (Industrial Scientific, Oakdale, PA, http://www.predictivesolutions.com/solutions/SafetyNet/) as standard practice prior to study initiation. To be eligible for inclusion, a site had to operate for >4 months from study initiation and have ≥30 workers at any one time (no maximum level of workers required). The sites within each pair were randomly assigned a treatment status of either control or intervention.
Treatment conditions
The intervention worksites implemented the B-SAFE program for 4–6 months (table 1). The program’s primary components were: (i) weekly worksite safety inspections; (ii) weekly feedback and communication; and (iii) monthly recognition and reward.
Table 1
Description of study sites. [I=intervention; C=control; R=renovation; NC=new construction]
The worksite safety inspections were conducted via site walkthroughs by a trained safety manager from either the general contractor or owner. The safety inspections provided weekly safety performance scores for the worksite and each subcontractor. Inspections were inclusive of all trades and tasks on-site and included both the safe (control measures) and unsafe (hazards) physical working conditions and practices. Although each site had a different inspector, all followed the same guidelines. The inspector entered all data into Predictive Solutions and denoted each observation by subcontractor. Once per week, investigators downloaded the inspection data and generated a weighted safety performance score (the percent of safe observations out of the total observations) for the overall site, and for the individual subcontractor companies (5, 14).
The weekly feedback and communication consisted of worksite posters and detailed reports distributed to each subcontractor on safety observations at the weekly foremen meetings. The research team distributed subcontractor-specific reports to the foremen that detailed all observations, both safe and unsafe, from the previous week. Large posters located in high visibility areas displayed a graph of the overall site safety performance score along with an adjacent list of the subcontractors’ recent scores. The poster contained an inspection score goal that ranged from 94.8–96.3% depending on the site. This goal was determined in a previous analysis by Sparer and Dennerlein (14) in which various methods were evaluated to best determine thresholds in leading indicator-based safety inspection programs. The final threshold goal utilized was determined to be fair, consistent, attainable, and competitive. The goal was the median of monthly safety performance scores over the previous 12 months from sites of similar size and scope from either the site owner or general contractor’s (based on how the pair was selected) inspection history (14).
The monthly recognition depended upon the overall site score for that given month. If the score exceeded the calculated goal, the whole site was recognized for their strong safety record with a catered lunch and participation in a raffle for either a one-month parking pass at a location near the worksite or a gas station gift certificate. If the score was below the goal, the research team conveyed this information to workers during foremen’s meetings and other whole site gatherings (such as stretch-and-flex).
The control sites consisted of the contractors’ standard safety programs along with a few posters with the B-SAFE logo only. Given the rigor of the data collection methods and high frequency of site visits required to do so, research team members were on both site types almost daily, leading to a strong presence at both.
Intervention efficacy evaluation
We used a mixture of quantitative and qualitative methods to evaluate the efficacy of B-SAFE. We considered this study an efficacy evaluation as it was conducted by study investigators in an environment in which the intervention and control conditions were highly standardized between sites (15). Worker surveys were completed pre- and post-exposure to the treatment and served to quantitatively assess changes in the study’s primary outcome of safety climate (figure 1). Due to limited time for survey data collection at the 10–15 minute coffee breaks, we used qualitative methods to assess all other constructs from figure 1 in a subset of workers.
Quantitative data collection
We invited workers on all sites to complete a pre-exposure survey at one of two times. For workers already on-site at study initiation, we invited workers at a study kick off meeting. For workers who started after the study began, we invited them during new worker safety orientations held multiple times per week. These orientations were mandatory and led by the general contractor. After collecting all completed surveys at intervention sites, we gave a 5–10 minute oral presentation that described the B-SAFE program. At control sites, workers were simply told that B-SAFE was a study of worksite safety and researchers would be on-site regularly to collect surveys. Workers aged 18–65 who could read and write English were eligible for the survey. We collected and compared names from survey respondents to track workers moving between sites.
We invited workers still on-site to complete post-exposure surveys every 30 days following their pre-exposure survey. We used a mixture of text messages and communication with on-site foremen and management to determine if a worker was still on the study site for the monthly follow up survey (16).
When we initiated the study in 2010, we made the decision to use a safety climate questionnaire developed by Dedobbeleer and Béland for use within the construction industry (17) given the importance of using an industry-specific scale to describe the safety climate (18) (Appendix A, www.sjweh.fi/index.php?page=data-repository). The questions were indirectly based on Zohar’s original 40-item and 8-factor scale (6, 17, 19). Gillen et al (20) used the Dedobbeleer and Béland items as a single factor to measure safety climate within a cohort of construction workers and found a positive association between safety climate and injury severity.
We modified the Dedobbeleer and Béland safety climate items in two ways. First, we modified the responses of the questions to reflect a Likert scale response. Second, based on inspection of the nine items, it became apparent that some items appeared to represent safety performance constructs such as safety training (items 6–7) and risk perception (items 8–9) rather than safety climate. Therefore, we performed an exploratory factor analysis using principal component analysis to assess the internal validity of the scale.
Factor analysis of the 9-item baseline data indicated that the items grouped together in two factors, a 7-item (scale items 1–7) and a 2-item (scale items 8–9). However, 2 of the 7 items (6–7) had factor loadings that were very weak (<0.50). We also had concerns about these items from a theoretical standpoint as they did not fit well with the definition of safety climate. Therefore, we did not include these items in our final safety climate scale. In addition, the Cronbach’s alphas for the various scale configurations were 0.71, 0.74, and 0.75 for the 9-, 7-, and 5-item, respectively (alphas measured on a scale of 0–1) (21). Given the empirical and theoretical strength of the 5-item scale, we selected it scale for our data analysis.
As Dedobbeleer and Béland did not provide a scoring rubric, we developed one where each item was given a value between 0–10 and then summed for a total score. As a result each item was equally weighted. Higher scores indicated a positive safety climate. As the number of response options per item in the original scale varied from item to item (some having four responses and some five), the point contribution reflected this. For example, if an item had five possible responses, the point contribution to the overall score would be 0, 2.5, 5, 7.5, or 10, whereas if only four response options were present, the contribution would be 0, 3.33, 6.67, or 10. If a minority of items were missing, the total score based on the completed answers was scaled to match the distribution of responses by the completed score.
The pre-exposure survey captured workers’ age in years, gender, union membership status, specific trade, job title, tenure in the construction industry in years, and highest educational attainment. We combined the responses of the two race and ethnicity questions to indicate non-Hispanic (includes Black/African-American, Asian, and Native American), White or other (includes Hispanic and respondents who indicated “other”).
Post exposure surveys included four intervention penetration questions: (i) Are you familiar with the worksite safety performance poster? Yes/no/not applicable; (ii) Are you aware of how your safety scores compare to other subcontractors? Yes/no/not applicable. (iiia) Have you received feedback from foremen or other site personnel on your company’s safety performance? Yes/no/not applicable. If yes, then (iiib): How does your foreman share information with you? Responses included: during weekly toolbox talks, one-on-one with workers, other, and does not share information.
We also tabulated the cost and time of implementing the intervention. These costs include the recognition lunches (food and raffle items), posters, flyers, and hardhat stickers. We also recorded the time it took to generate the safety scores and provide site feedback.
Quantitative data analysis
We first completed a bivariate analysis comparing the change in safety climate and worker demographics between control and intervention sites using Chi-squared tests of homogeneity for categorical variables and t-tests for continuous variables.
We then generated three mixed-effects regression models with the difference in pre- and post-safety climate score as the dependent variable, and treatment status (intervention or control) as the independent variable. For the first model, we included a worksite variable as the random effect in the model to account for the site-to-site variability in safety climate scores. For the second expanded model, we included a matched pair variable as a fixed effect based on our block randomization procedure. For the third model, we expanded the second model to include categorical variables for the month the worker started on-site and the total amount of time the worker spent on-site. This third model also included variables selected via stepwise variable selection technique from the worker demographic variables that differed between the control and intervention sites in the bivariate analysis with p-value less than 0.2 (table 2). All data analyses were completed in SAS version 9.3 (SAS Institute Inc, Cary, NC, USA) and were considered significant at P<0.05.
Table 2
Worker characteristics at control and intervention sites. [GED=general education development; SD=standard deviation]
Qualitative data collection and analysis
At the end of quantitative data collection, we conducted one worker focus group per site, which occurred during an extended lunch break (half hour lunch break plus approximately 15 minutes of working time). A representative from the general contractor publicized the time and location of focus groups. Focus groups were open to all workers, with the first workers to arrive participating. Each focus group had six to eight participants, spanning the various trades and job titles. We followed a discussion guide that included questions on overall perceptions of site safety and related constructs (eg, management commitment to safety, teamwork, and safety awareness). Example questions included: Did you feel that management cared about safety? Did you feel that this was a safe site and why or why not? Have workplace safety conditions changed over the past few months here? Three research assistants coded and analyzed independently the recorded transcripts for thematic content using Atlas.ti (version 7).
Results
Quantitative data
Study population and response rates
Seven general contractors/owners were invited and four agreed to participate. These four provided ten sites, of which eight agreed to participate (figure 2, table 1). Those that declined cited tight work schedules and/or concerns from the property owner. In total, 1289 workers completed the pre-exposure baseline survey, with a response rate at intervention sites of 71% and control sites of 81%. The study sample included only those workers with both pre- and post-exposure (N=615). The response rate for the post-exposure follow up survey for eligible workers (those on-site at the time of follow up) was 88% at intervention sites and 86% at control sites. Of the 615 people, only 9 were on multiple worksites (indicating a contamination rate of approximately 1.5%).
The size of worksites and worker characteristics differed between the control and intervention sites despite randomization (table 2). All intervention sites were approximately twice the size of the control sites. Workers on intervention sites differed from those on control sites in terms of age, industry tenure, trade, and job title, as well as in terms of month started on-site and total time on-site (table 2). Baseline safety climate scores were also higher on control sites (P=0.040).
Safety climate
The mean safety climate score of the intervention sites increased by 0.5 points (1%) between pre- and post-exposures, compared to the control sites, which decreased by 0.8 points (1.6%) (table 3). Three out of four intervention sites showed a positive increase. The sites that started off with the lowest mean pre-exposure scores (Pair 1-Intervention Site, and Pair 3-Intervention Site) had the largest increases [1.1 (2.2%) and 1.9 points (3.8%), respectively].
Table 3
Summary of pre- and post-exposure mean safety climate scores (0–50 with increasing positive safety climate) across the eight study sites. [SD=standard deviation]
This effect increased and became significant in the mixed-effects regression models (table 4). The variance between the sites, while not statistically significant, was greatly reduced with the addition of the pair variable in Model 2. The third model included adjustments for worker trade, job title, age, race/ethnicity, month the worker started on-site, and total number of months the worker was on-site. We ran similar regression analyses on the full nine item scale. The results were similar in direction, although slightly weaker in magnitude than the results of the five item scale.
Table 4
Results of Mixed Effects Regression Models [SE=Standard error].
| Intervention effect estimate | N | SE | P-value | |
|---|---|---|---|---|
| Model 1 – Unadjusteda | 1.54 | 604 | 0.80 | 0.06 |
| Model 2 – Adjustedb | 1.58 | 604 | 0.59 | 0.01 |
| Model 3 – Adjustedc | 1.64 | 600 | 0.63 | 0.01 |
Intervention penetration and cost
At intervention sites, workers were more likely to be aware of how their safety performance compared to other subcontractors and to receive/share feedback from their foremen/with their workers (table 5). The additional cost of running the B-SAFE program for five months was $3055 plus one work hour per week per site, which represented the time for a staff member to compile the scores and the reports (table 6). This cost estimate assumes that weekly safety inspections are already part of the worksite health and safety program.
Table 5
Penetration B-SAFE program components at intervention and control sites.
Table 6
Estimated costa of running B-SAFE for five months.
a The cost estimate relies on the following assumptions: (i) the site has a health and safety program that includes frequent safety inspections and entering data into the Predictive Solutions database; (ii) the person running the program is a trained health and safety manager; (iii) the intervention is run on a worksite for five months; (iv) the site surpasses the safety performance threshold three out of five months; (v) there are 75 workers on the site during recognition lunches.
Qualitative data
The individuals who participated in the focus groups (intervention: N=33; control: N=24) had a mean tenure in the construction industry of 17.5 years and were from trades including pipefitters, electricians, carpenter, ironworkers, and laborers.
At both control and intervention sites, workers noted common themes, including good communication, management commitment, teamwork, and safety awareness. Workers at intervention sites noted positive changes in these themes; whereas workers at control sites mentioned few differences: “No changes [during the B-SAFE study]—its safety first from day one.”
Safety communication appeared to improve at intervention sites: “[B-SAFE] helped safety-wise definitely, to be cautious of other people and what’s around you, and that’s huge. Communication is key between the trades. First couple of times you do it looks like you’re a jerk but now everyone sees the reason why and are looking out for everyone’s safety.”
Safety awareness and teambuilding were expressed more frequently and in a more positive light during focus groups at intervention sites compared to those at control sites. One worker at an intervention site noted: “With ladders, normally you’d just want to get it done, and you’d take the extra foot on the ladder, now guys are conscious to go get a higher ladder. You see the guys making the change. It makes you more aware. Now, you’re being more conscious of everything else.” A different worker at an intervention site noted: “The trades were working together with the B-SAFE program, and other trades were watching out for everyone else. Normally they would never do that, but now I see talking amongst the trades—this came from the program.”
Positive reinforcement of safe work and collaborative competition were unique themes identified at the intervention sites. Collaborative competition was defined as workers expressing interest in improving their own contractor’s safety performance score in order to beat the other subcontractors on the worksite, as well as improving the whole site safety score. “No sub wanted to mess up and cost the other guys—who were trying hard—the lunch.” Positive reinforcement was noted by a foreman, “[It] felt good to finally get a piece of paper in a meeting to say I did something right.”
Discussion
The goal of this study was to evaluate the efficacy of B-SAFE, a safety communication and recognition program, on a set of worksite safety measures. The results indicated that B-SAFE had a positive impact on these site safety measures, leading to an improvement in safety climate of approximately two points on the intervention compared to control sites. Qualitative data also indicated a higher occurrence of positive safety-related themes of teamwork and increased awareness at intervention sites when compared to control sites. These improved safety metrics may lead to reduced rates of work-related injury.
Our results were similar in magnitude to changes Zohar and Polachek (22) observed in safety climate. Their supervisor communication intervention had an effect size of 0.15 on a 5-point scale, a 3.8% change, compared to the B-SAFE study effect size of 1.64 on a 50-point scale, a 3.28% change. The slightly smaller effect size of the B-SAFE study can also be attributed in part to the high variability of the construction environment, compared to the stable environment of manufacturing, and the added challenges faced of running and evaluating a program when the population of workers changes constantly (16).
Studies in other industries have focused on relationships between injury and safety climate (23), as well as associations between mediating or modifying factors on the pathway of safety climate and injury such as safety behavior, employee safety control, and safety leadership (7, 24). Although our study was not powered to detect difference in injury and further research is needed to examine the true effect of B-SAFE on injury outcomes, the observed 1.6 effect size still might have practical significance. The approximate change represents close to a 16% increase in the available range of positive change (10 points out of 50). There are some limitations that should be acknowledged with regard to the use of the Dedobbeleer and Béland scale. As we have previously described (25), there are concerns regarding reference groups in the scale, as the referent category changes between the job itself, the worksite, and the company. These issues might result in increased variability in the safety climate measurement; however, this would impact both the intervention and control groups equally, thus resulting in an effect estimate biased towards the null. There are also concerns regarding some items which may reflect individual rather than shared perceptions of safety, however these dropped out after our factor analysis and were not included in our final 5-item scale.
Conducting a randomized trial of a safety intervention on highly dynamic construction sites proved to contain many challenges, which place additional limitations on our conclusions. First, there were significant differences in the site populations between the control and intervention sites. The small number of sites allowed for the imperfect randomization. A priori and based on our understanding of the local construction industry, we believed that there would be more variability between sites of different general contractors/owners than within a general contractor/owner. We therefore aimed to find two sites within each general contractor/owner that were of similar size and trade make-up and block randomized within this pair. However, the reality of conducting research on active construction sites meant that we were limited to the sites that were available during our data collection period in order to have concurrent control and intervention sites. As a result, there was a substantial size and demographic makeup difference between the sites that were randomly assigned to either intervention or control status. To account for these differences, we therefore added site- and individual- level variables to our statistical modeling. Adding these variables increased the effect estimate and statistical significance. While there was some site-to-site variability in the mixed model, the matched pair design helped to refine the effect estimate.
Second, the workers on construction sites come and go frequently with about 50% remaining on-site >30 days (16). The data used in this analysis were collected from workers who were on-site for >30 days. The conclusions therefore may not be reflective of all workers and may exhibit a form of selection bias. The workers who are on-site >30 days are different in their distribution of trade, job title, race/ethnicity, and baseline musculoskeletal pain than workers who are on-site <30 days (16). The surveys analyzed in this study may not reflect a population representative of the true worksite composition, with those captured tending to be healthier (26). Within our sample, we addressed this issue of potential bias by controlling for time-varying parameters (total time on-site and month started) in our analysis. When we controlled for length of time on-site the effect estimate increased and standard error decreased, suggesting that the original effect estimate might be an underestimate.
Another limitation could be the movement of workers between study sites; however, contamination was very low (1.5%). It is possible that workers on whom we do not have data (either because they declined our baseline survey or did not provide contact information in the follow up survey to allow for follow up) moved between sites. However, any undocumented contamination would, if anything, likely bias our results towards the null.
Worker mobility also required that the pre-exposure safety climate score was determined on a worker’s first day on the jobsite, where they may not have fully formed their safety climate perceptions and, as a result, lead to increased variability in the measurement. However, this increased variability would equally affect both the intervention and control sites. If any, the bias on the results would be non-differential towards the null (27).
The conclusions are limited to commercial construction sites with pre-existing safety programs. The findings in this study indicate that B-SAFE can have a positive impact on site safety; however, it was tested on sites that were assumed to already have strong safety program in place based on the owner/general contractor. B-SAFE is not a standalone safety program; it was designed to be a low cost add-on to an existing health and safety program of high quality that includes a robust safety inspection program. The companies and sites included in this study had sophisticated systems of safety as indicated by their use of Predictive Solutions and the high safety climate scores. In addition, the sites were all medium to large commercial sites and heavily unionized.
Induced bias due to regression to the mean is another possible limitation. This occurs when extreme values tend to be followed by more typical values and is of particular concern when the sample is small and has not been randomly selected. To minimize this bias, we determined site-type randomly and within each site were able to collect data from a majority of respondents. While we cannot remove this potential bias completely, our methods have reduced the possibility of such induced bias.
In conclusion, B-SAFE led to many positive changes on the worksites, including an improvement in safety climate, awareness, teambuilding, and communication. It was a simple, low cost intervention that the construction industry can use to improve safety climate on worksites. Simple programs that engage all workers through strong communication infrastructures may have a positive impact on overall worksite health and safety.
