Scientific basis of ISO standards on biomechanical risk factors

This paper reviews the processes used in the development of ISO standards related to occupational safety and health and compares the processes to those typically used to develop evidence based medical guidelines on sources of bias and transparency. Among other purposes, companies and regulatory agencies from around the world often adopt International Standard Organization (ISO) standards to determine acceptable practices, equipment and criteria for preventing occupational injuries and illnesses. ISO standards are based on a consensus among individuals who participate in the process. This discussion paper examines the scientific process for the development of several ISO standards on biomechanical factors, comparing it with processes used by other professional organizations, including scientific committees working on the development of clinical guidelines. While the ISO process has value, it also has clear limitations when it comes to developing occupational health and safety standards that should be based on scientific principles.

The International Standard Organization (ISO) is an independent, non-governmental, international organization that brings together experts to share knowledge and develop voluntary, consensus-based, market-relevant standards supporting innovation and providing solutions to global challenges (1). The ISO's main deliverables are the so-called "ISO standards", which are prepared by technical committees based on a framework protocol (2). According to ISO, "A standard is a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purposes" (3). The more than 21 000 ISO standards deal with many unrelated topics, ranging from soaps to spacecraft, MP3 to coffee (4,5).
Although ISO standards are not conceived as a part of a national or international regulatory process, many countries have developed policies to facilitate, or even enforce, their application. For example, Brazil has recently adopted the ISO ergonomics standards (discussed below) as a regulation. In the European Union, the so-called "new approach to technical har-ISO standards on biomechanical risk factors monization" is based on, but not limited to, indirect reference to ISO standards in the EU Directives; thus, some of the EU Directive is, at least partially, based on standards published by ISO (6). As a result, the technical and scientific communities generally consider the application of ISO standards as a good practice. However, ISO standards are actually "voluntary agreements" not necessarily developed according to a rigorous scientific process (7). Thus, problems might arise when ISO standards are adopted as governmental policy on topics that should be evidence-based. This is relevant for health professionals because some ISO standards cover aspects of the work environment and workplace health and safety. For example, there are ISO standards on protecting workers from hand arm vibration, the design of safety glasses, workplace noise, and so on. If these standards are to be adopted as workplace health and safety policies, they should, to the extent possible, follow an evidence-based scientific approach.
As an example of ISO standards that cover aspects of the work environment and workplace health and safety, the Anthropometry and Biomechanics Subcommittee (SC3) -part of the ISO Technical Committee 159 Ergonomics (ISO/TC 159/SC3) -prepared 26 published standards, among which are to be found some recommended occupational limits of exposure to ergonomic risk factors (eg, ISO 11228-1 presents a risk-assessment model to prevent musculoskeletal disorders among workers lifting and carrying loads) (8,9). The validation of a risk-assessment method for biomechanical risk factors is essentially a multistep scientific process. At first, rigorous laboratory and epidemiologic studies are conducted and replicated by independent research groups. Then, all available evidence should be systematically evaluated and synthetized through a transparent review process. For instance, the American Conference of Governmental Industrial Hygienists (ACGIH) proposed threshold limit values to assess the risk of hand/wrist disorders due repetitive/forceful movements that have been evaluated in large prospective studies and many smaller cross-sectional studies, and the findings have been published in international peer-reviewed journals (10,11); the available studies have been analysed in a systematic review published in a peer-reviewed journal (12). Furthermore, when multiple options are available for risk assessment methods, the choice of the risk-assessment method to be recommended should be based on systematic comparative evaluations considering both scientific and technical issues (13). Scientists have become concerned by the diffusion of standards that are not the result of a rigorous synthesis of the available scientific evidence. In 2001, Fallentin and colleagues reviewed some technical standards (ISO, European Committee for Standardization) on physical workload and the exposure limits and commented "…technical standards on ergonomics and physical workloads, for example, CEN (Committee for European Standards) and ISO (International Organization for Standardization) standards, continue to present very specific exposure limits and equations to predict acceptable workloads. Due to limited legal implications, the CEN and ISO standards have been "allowed" to present very specific and rather unsupported limits without much public debate. The question of scientific validity is essential for all researchers involved in the study of work-related musculoskeletal disorders…" (14,(247)(248).
An example of the application of a transparent process for reporting of scientific evidence are the widely known and accepted guidelines developed by the EQUATOR network (15). The development of most medical practice guidelines follows this approach using the Appraisal of Guidelines, Research and Evaluation (AGREE II) checklist to evaluate quality and bias (16). The AGREE II checklist considers key factors that may lead to bias, such as the reporting of members involved in the authorship, the reporting of the competing interests of those members, the methods applied to review the evidence, the methods to formulate recommendations, the external review process, the process for updating the guidelines, and funding sources. This sound approach to scientific evidence is currently a requirement in many fields (evidence-based medicine, evidence-based policy, evidence-based legislation, just to name a few).
To determine the degree to which workplace healthand safety-related ISO standards followed an evidencebased scientific approach, we applied the AGREE II checklist to ISO standards dealing with ergonomic issues (i.e. ISO 11226, 11228-1, 11228-2, and 11228-3) and discuss the possible consequences of the lack of compliance with the AGREE II checklist (8,(16)(17)(18)(19).

Selection of ISO standards and the review process
In August 2017, we read the "scope" paragraphs of all ISO documents produced by the Anthropometry and Biomechanics Subcommittee (ISO159/SC3) to identify guidelines on risk assessment methods for occupational biomechanical factors. Published standards (the main deliverable from ISO) were identified along with other types of documents -such as technical reports -as possible sources of additional information. We selected ISO standards 11226 (static postures), 11228-1 (lifting and carrying), 11228-2 (pushing and pulling), and 11228-3 (handling low loads at high frequency) for review, as they cover important biomechanical risk factors in the workplaces. ISO technical report TR 12295 -application document for international standards on manual handling (ISO 11228-1, ISO 11228-2 and ISO 11228-3) and evaluation of static working postures (ISO 11226) -and ISO Armstrong et al technical specification TS 20646 -ergonomics guidelines for the optimization of musculoskeletal workload -were consulted as further sources of information, as they present additional details on the application of relevant standards. All the consulted documents were defined "in effect" (ie, published and not withdrawn) by ISO at the time this paper was drafted. These documents are referred to as the "ISO ergonomics standards" in this paper.
AGREE II is a widely used approach for assessing the methodological quality of practice guidelines (16). The checklist comprises 23 items (each presenting specific reporting criteria) grouped into the following six domains: (i) scope and purpose; (ii) stakeholder involvement; (iii) rigor of development; (iv) clarity of presentation; (v) applicability; and (vi) editorial independence. The AGREE II guidelines are mainly aimed at being applied prospectively during the drafting process of a practical guideline; however, they can also be used retrospectively as a quality assurance step. Of note, the checklist was developed to be sufficiently flexible to fit different contexts, independently from the specific protocols and methods used to support the practical guideline (16). A priori, we decided to focus on three of the six domains of the AGREE II checklist (eg, stakeholder involvement, rigor of development, and editorial independence) as they convey fundamental information on bias and the scientific bases of any practical guideline.

Application of AGREE II checklist to ISO ergonomics standards
The compliance of the selected ISO ergonomics standards with three domains of the AGREE II checklist is presented in table 1. The ISO standards do not fulfil most of the AGREE II criteria. One of the most relevant issues is the absence of information about the subcommittee members (with the exception of the chair): their identity is undisclosed and their scientific profile is not described. Stakeholder involvement is a cornerstone of ISO procedures (2) and identified as an important criterion on the AGREE II checklist, but the involvement of key stakeholders (eg, labor authorities, companies, ergonomics professionals, knowledgeable scientists) is not clear. One of the potential caveats of every risk assessment method is the level of knowledge/expertise necessary to apply it properly and efficiently in a realworld setting (13). The ISO ergonomics standards do not present specific information on this aspect.

Rigor of development of the ISO ergonomics standards
The production of the ISO ergonomics standards differed substantially from the writing of evidence-based practical guidelines. According to the limited information provided in the published documents, the ISO Target population preferences and views Stakeholder engagement is a part of ISO procedures (2), but the role of stakeholders in the development of each standard is not described. It is not specified if, and how, feedback from stakeholders were integrated in the standards. Possible stakeholders for ergonomic guidelines are not defined or identified.

Target users
The intended guideline audience (ie, who should perform the risk assessment) is not described. Instructions on how to apply the risk assessment methods are provided within the standards and in additional technical reports (e.g. TR 12295).

Domain 3: Rigor of development Search methods
A systematic literature search is not described in the standards. It is not clear if a (systematic) review of available evidence was conducted. Evidence selection criteria Inclusion or exclusion criteria of studies are not described. It is not clear how data were extracted from the original articles.

Strengths & limitations of the evidence
The review of the studies are not based on a formal or informal appraisal of evidence and bias. The quality of studies referenced are not assessed. Heterogeneity between studies is not evaluated or addressed. The guidelines do not present quantitative information on reliability and validity of risk assessment methods. Internal and external validity is not discussed.

Formulation of recommendations
The methods used to formulate the recommendations are not described. Disagreement among subcommittee members is not presented. It is unclear how disagreement, if present, was handled. Link between recommendations and evidence There is no process for linking evidence to recommendations (eg, grade of recommendation based on available evidence/expert opinion). Not all the recommendations are directly supported by referenced studies. Recommendations are not preceded by a summary of evidence.

External review
No process for external review is identified. Updating procedure The ISO website clearly defines the stages of development (eg, publication, review, withdrawal) of each standard according to a harmonized coding system (available at www.iso.org/stage-codes.html). All the standards are meant to be reviewed or confirmed every five years. All changes to ISO standards are tracked.

Funding body
There is no statement about funding to support subcommittee members.

Competing interests
A competing interests statement of all subcommittee members is not available to the public. It is unclear how competing interests are or might be addressed.

ISO standards on biomechanical risk factors
ergonomics standards were not based on a systematic search and appraisal of the available literature. It is not clear why the ISO subcommittee preferred one method of risk assessment over others. For instance, the ISO 11228-3 identified three detailed risk assessment methods for repetitive hand exertions at high frequency: OCRA (a concise index for the assessment of exposure to repetitive movements of the upper limbs) (20), ACGIH hand activity level (HAL) (21), and the Strain Index (22), but preferred the OCRA methods without providing a scientific basis or comparison (eg, intra-and inter-observer reliability, strength of association with musculoskeletal disorders (MSD), etc.) even though such comparisons are available in the literature (13,23). As a result, some statements in ISO 11228-3 appear to be based on personal opinions and are in contrast with scientific evidence from the literature. For instance, the ISO standard includes a statement "in many epidemiological surveys it (OCRA) has shown itself to be well related with health effects (such as the occurrence of UL-WMSD [upper limb-work related MSD)]" (13). This statement was not supported by a well-designed epidemiological study in 2007 when the ISO standard was published (19). Indeed, in 2010, Takala and colleagues noted the absence of longitudinal studies on the association between the OCRA index and the risk of MSD. They also pointed out the absence of studies on the repeatability of the OCRA method (13). The basis for formulating the recommendations of the ISO ergonomics standards did not follow the AGREE II criteria. Relevant scientific studies were not evaluated for strengths and weaknesses and assigned a formal grade. Without such a review, it is not possible to establish relative merits or priorities. Also, when dealing with exposure assessment methods, a practical guideline should try to establish achievable goals; the ISO standards do not provide any information on the reduction (or increase) of the risk of MSD expected for any given level of exposure. Hence, compliance with ISO standards may not provide confidence that occupational risks are adequately addressed. Furthermore, statements in the ISO ergonomics standards are not referenced and linked to a scientific study that supports the statement as would be expected in a scientific paper or review. Although ISO standards are not conceived to be part of the scientific literature, the transparency of the creation process would be greatly enhanced if the scientific rationale was presented in supporting documentation.
A strength of the ISO standard process is the thorough and transparent flow of the updating procedure, such that the users have clear knowledge of the state of the standard writing timeline.

Authorship of the ISO ergonomics standards
Scientific reviews, public health and medical guidelines, and scientific papers are the responsibility of the authors who are clearly identified in the publications. Since the scientific process is an open one, anyone reading the publication can check the competence and bias of the authors by retrieving previously published papers, resumes, institutional information, and so on. In addition, authors of guidelines are required to report any conflict of interest (financial and other interests), which may be related to the issue on which they are writing. Competing interests are a major concern in the scientific community. The ISO standards, on the other hand, do not publish the names and affiliation of the members and their possible conflicts of interest, and there is a lack of transparency on funding. There may be advantages for keeping the ISO authorship anonymous. Committee members may change frequently, some people may not participate if their name is publically listed, and independence may be facilitated by hiding the identities. However, this approach is counter to transparency and the AGREE II recommendations.

External review of the ISO ergonomics standards
The ISO ergonomics standards did not undergo an external peer-review by key stakeholders, relevant professional societies, or interested scientists. Therefore, the ISO ergonomics standards should not be considered as widely accepted by these other bodies. All other medical practice guidelines, or public health standards, undergo external reviews as recommended by AGREE II. The quality of evidence-based guidelines is enhanced by the review process. The process may not be perfect, but it ensures that scientific endeavors are a self-correcting and self-improving activity. After the publication, it is the scientific community at large who will corroborate (or not) the recommendations of guidelines by direct critique, by independent reanalysis of the same data (thanks to the "open data" initiative), or by new studies. The issue of the quality of public health guidelines is of paramount importance; no one wants effort and resources wasted on a large scale on actions that have dubious public health value. This important external review process is not undertaken during the development of ISO standards.
It is noteworthy that the scientific community has directed relatively little attention to the ISO ergonomics standards. At the time of this writing, PubMed (search string: "11226" AND ISO) identified only three papers which reference ISO standard 11226, none of which is a validation study or otherwise provides support to the limits and other recommendations included in the standard. Also, a literature search for articles citing Armstrong et al ISO standard 11228 (search string: "11228" AND ISO) identified only 16 pertinent papers (9 of which originating from the same group of Italian coauthors). Again, none of the retrieved papers is a formal validation study or otherwise provides support to the reference values included in the standard.

Stakeholders' perception of ISO standards
The application of a risk assessment tool to a real workplace requires a multidisciplinary approach and can have relevant social implications. Possible stakeholders range from scientist to technicians, from employees to employers, from labor unions to customers' associations, from administrators to policymakers. In the presence of several, and possibly contrasting, points of view, there is the potential for a relevant conflict of interest. One of the possible consequences is a misleading presentation of scientific evidence aimed at inhibiting an unbiased debate. Indeed, the public opinion might perceive a technical standard published by an international association as objective and not subject to further evaluations by stakeholders. As an interesting example of the pitfalls of this process, we can mention the controversy on the adequacy of an ISO standard to guide tobacco products regulatory policies; a lack of transparency in the production process of the standards was reported by stakeholders and Bialous and Yach stated in a peerreviewed manuscript that "no health claims can be made based on ISO's tobacco product standards" (24). Hence, it is imperative to ensure a transparent presentation of scientific evidence and to define clearly the domains, and the limits, of the evidence-based decision-making. At present, the ISO standards have failed to cope with these aspects. As an example of the consequences of the unclear presentation of the rationale and the scientific bases of a technical standard, it is worthwhile to mention the ongoing debate on the future ISO 45001. The aim of this standard, under publication, will be to "enable organizations to manage their OH&S [occupational health and safety] risks and improve their OH&S performance" (25, p1). Apparently, the scientific community devoted little attention to the document; we identified only one brief commentary in the Spanish language citing the standard (PubMed search string: "45001" AND ISO) (26). However, other stakeholders identified the potential pivotal role of such a standard and expressed concerns. In particular, the European Trade Union Confederation (ETUC) published a resolution stating that: "The ETUC along with the ITUC (International Trade Union Confederation) is concerned that the proposed International standard, ISO/DIS 45001 will, in many countries, represent a risk for the advancement of the legal framework promoting good health and safety at the workplace, if it is adopted in its current form" (27).
This viewpoint obviously implies a complex discussion that is beyond the purpose of our paper; however, any judgment on this topic would largely benefit from the clear identification of what is evidence-based. In addition, this example highlights how the scientific community has not paid enough attention to a topic that is perceived as fundamental by the European trade unions. Such a contrast is not unusual in our field; as we highlighted before, the ISO ergonomic standards are seldom cited in the scientific literature. This fact has two important implications: (i) it is not clear whether the scientists generally accept the methods proposed by ISO or not; (ii) ISO standards are not periodically discussed in the light of cumulating new evidence. In our discussion we have focused on the some critical issues in ISO ergonomic standards. However, we also have concerns related to scientists' conduct in the face of ISO standards. This general lack of interest might have contributed to the spread of technical practices that are not line with best available evidence. As an example of a virtuous approach to ISO standards, we would like to recall the important contribution provided by Professor Micheal J Griffin. As a leading scientist in the study of human responses to vibration, he also included among his interests the critical interpretation of ISO standards related to vibration (eg, 28,29). This kind of approach would be unsurprising in other fields related to occupational health, such as the study of occupational carcinogens; indeed, there are many peer-reviewed papers aimed at discussing the appropriateness of classification on carcinogens presented by authoritative international agencies. We believe that the same process should happen with ISO standards and we hope that researchers in occupational health will increase their interest in discussing any proposed standard related to occupational risk assessment.

Concluding remarks
There are several methodological aspects that make the ISO process very different from the processes used for developing medical practice guidelines, including public health guidelines. The purpose of public health or medical practice guidelines is to recommend approaches to treatment or prevention of injuries or diseases that are based on systematic and transparent scientific reviews of the literature. An ISO standard has a different purpose, in that it is an effort by a self-identified committee of interested people to agree on "how something should be made" in order to facilitate exchange of goods, services, or other similar endeavors. This is a key distinction between a scientific review on a topic and an ISO standard on the same topic; the former will have to abide by the accepted peer-review process for scientific publications and development of recommendations whereas the ISO standards on biomechanical risk factors latter can use any approach that will suit the purpose, without having to take into account scientific method, evidence or even transparency.
The development of the ISO ergonomics standards reviewed (eg, 11226, 11228-1, -2, -3) did not involve transparent and evidence-based scientific review processes that are widely used in public health and in the healthcare field. The names and affiliations of the authors of the ISO standards were not identified (except for the chair), and there was no review of conflicts of interest. It was not evident that critical stakeholders, who will be impacted by the standard (eg, labor organisations and safety professionals) or even scientific specialists in the field (eg, research ergonomists and epidemiologists) were represented. The methods used for selecting the recommended force limits and risk assessment tools were not presented. Some risk assessment methods are recommended over others without providing an explanation of the criteria used to differentiate them. Overall, the lack of an evidence-based approach leads us to recommend that the ISO ergonomics standards should not be adopted as health policy by companies or governments.
Transparent and scientifically supported methods have been used by national and international organizations to develop work-place health and safety standards. For example, the ACGIH lists the committee members and considers their conflicts of interest. The ACGIH threshold limit values (TLV) include published background documentation that reviews the literature and provides a basis for the selection of action limits or TLV. Even with this greater transparency and evidencebased approach, ACGIH declares that the TLV "are not developed for use as legal standards and ACGIH does not advocate their use as such" (21, inside cover). The World Health Organization (WHO) has adopted another method for producing transparent and quality evidencebased guidelines (28). The method is based on first developing systematic reviews and then translating the evidence into recommendations based on the GRADE approach (29). The method has been shown to be resistant to commercial interests (30).
Safety practitioners and regulators might perceive ergonomic standards as highly accurate for safeguarding workers' health because most of them include exact formulas to assess the level of exposure or risk (13). However, cut-offs of continuous variable (eg, times, angles, or loads) are usually based on simple a priori classifications; for instance, scores usually range from 0-10 and quantitative measures are classified on rough discrete scales. Thus, the formulas are actually based on approximations, and their discrimination value may be low. This fact is the obvious consequence of the difficulty in collecting highly accurate measures with observational methods and should be clearly acknowl-edged to avoid an excessive confidence in observational methods due to "false accuracy".
The purpose of this Discussion Paper is not to critique the ISO process, per se, but to clarify that it is based on the opinions of the subcommittee's participating members and not on evidence-based scientific methods. Therefore, if ISO standards are referenced as an approach for the prevention of work-related injuries or illness, they should be used with caution. As repeated in the ISO documents, ISO standards are intended to be voluntary. Mandatory policies adopted by governments or companies for the prevention of workplace injuries or illness should, instead, be based on evidence-based scientific methods.
Finally, we call for an increase in the attention scientists devote to the published ISO standards; in particular, standards dealing with risk assessment at the workplace should be discussed in the light of best available evidence.