Inconclusive cancer epidemiology - Hernberg, Sven (Scandinavian Journal of Work, Environment & Health, 1998)
How often or not is the main message of an original epidemiologic article the old cliché "further research is needed"? Sometimes the literature is even so dominated by inconclusive or contradictory articles that comprehensive reviews have to come to the same conclusion. In this issue we see two examples. One of them states that, although much research has been published, it is not possible to clearly identify environmental or work-related etiologies for ovarian cancer (1). Except for tobacco smoking, the same appears to be true for pancreatic cancer (2). In an earlier review a similar conclusion was reached for breast cancer (3). There are several other cancers for which environmental or occupational etiologies have been suggested but for which the evidence is inconclusive. Stomach, colorectal, and prostatic cancer can serve as examples. On the other hand, there are indeed cancers for which strong environmental etiologic factors have been identified by epidemiologic research, for example, lung cancer, sinonasal cancer and primary liver cancer. Many hundreds of chemical, physical, biological and life-style factors have been reviewed by the International Agency for Research on Cancer, but its expert teams have only been able to classify fewer than 100 chemicals, work processes, or other environmental factors as human carcinogens. Several others have been classified as probable or possible carcinogens, but in the vast majority of the evaluations the evidence was inadequate. And even when carcinogens have been identified, they usually explain only part of the etiology of a particular tumor. The problem is usually a lack of literature in general, and epidemiologic literature in particular, and even if the latter exists, it is all too often limited, confusing, contradictive or inconclusive. Why is it that a review of the epidemiologic literature can so seldom reach definite conclusions? There are several reasons. One could be that no major environmental or work-related risk factors for the cancer in question really exist. When this is the case, some studies may turn out "positive" and others "negative" by pure chance, especially if the population under study is small. Alternatively, environmental or work-related risk factors could be too weak (biologically) or the exposure levels too low to be detectable by the rather crude tool that epidemiologic research provides. However, the well-known geographic differences in cancer occurrence suggest that these alternatives are not sufficient explanations. The literature may also be inconclusive due to the methodological problems inherent in epidemiologic research, weaknesses that even the best of researchers cannot overcome. Such problems include lack of retrospective exposure data, difficulties in finding large enough study populations, and so forth, as discussed later. A further explanation may be our poor understanding of the mechanism of action of many carcinogens, for example, are early or late stages of carcinogenesis involved, or both, and what is the interaction with genetic factors. Knowledge of such mechanisms would help focus studies on vulnerable population groups and the most informative time windows. Different histological subtypes of cancers should ideally also be studied separately, but this approach often requires much larger populations than those available. These are convenient explanations for epidemiologists because they blame "natural" factors and not the skills of the researchers. But is this the whole truth? No, it is not. Too much poor and unnecessary research is still being carried out and published. This was, in fact, one of the conclusions of the review on ovarian cancer (1). Although the general rule in science is that only new and original ideas should be elaborated into a research project, epidemiologic research is somewhat different. It is inexact and prone to errors, and therefore other researchers' findings must be corroborated or disproved. Clarification is especially needed when the literature is inconclusive or contradictory. But this need for clarification does not give us a green light to initiate whatever project whenever. The key issue is whether a new study really will be better than earlier attempts. Then, and only then, does the problem have a chance of being solved; otherwise the new endeavor only adds to the confusion. Sometimes a new project can indeed beat earlier ones, for example, if the researchers have access to a unique population with good exposure data, if their competence is high, or if they can apply new and better methodology, for example, in exposure assessment. If so, there is good reason to carry out the study. More often however replicating earlier studies does not lead anywhere. Then replicative research is not warranted. Even if the idea is original, scientifically sound, and important, many "natural" shortcomings typical of epidemiologic research may spoil a project. Very often the population available is too small, especially when it comes to the number of truly informative individuals (ie, those who have been highly exposed long enough and many years ago). The situation is especially problematic whenever the turnover of the work force is high and exposures occur mainly in small workplaces. In the era of meta-analysis, a small population is not completely useless, however, provided the study is otherwise sound. Nevertheless, small studies are in general uninformative when regarded alone. Another "natural" shortcoming is a mixed exposure pattern. Mixed exposure patterns can render the results of a study uninterpretable. Such patterns are a fact of life, not a flaw in study design, but prudent researchers avoid such constellations if the issue is the carcinogenicity of a single agent. Sometimes this is not possible, for example, when the issue is exposure to pesticides or cytostatic drugs. The exposure could also be too low to cause detectable effects, or it could affect almost everyone (eg, dietary factors, food residues); in such cases sufficient exposure contrasts are difficult to achieve. Measuring exposure accurately is necessary in epidemiologic research, and failure to do so has ruined many studies. The problem is that reliable retrospective exposure data very often do not exist. In the absence of such data, the exposure must be assessed. However, retrospective exposure assessment is an educated guess, even at best. Using occupational titles as the only proxy for exposure is far too crude an approach, with or without register linkage, as the authors of one of the reviews in this issue points out (1). All too many studies have used this approach, which has led to much confusion. Such studies should no longer be published. Using crude exposure data or poor proxies leads to nondifferential misclassification of exposure, which is strongly effect-masking in case-referent studies. A nondifferential misclassification rate as low as 10% lowers the estimate of the odds ratio by almost half (4). In case-referent studies focusing on distant time periods, the misclassification rate is often probably even far higher. How much faith can then be put into such odds ratios, and how can odds ratios obtained from different studies be compared if the degree of misclassification in various studies is different or if it cannot even be assessed? Likewise, how can rate ratios or standardized mortality ratios (SMR) of cohort studies with poorly documented and probably highly different exposure levels be compared in a review? It has long been known that SMR values from different studies should not be directly compared, because they have not been mutually standardized. Still, they are compared, without hesitation, in most meta-analyses. Different levels of exposure can of course in principle be accounted for, but how can one do this when exposure data are missing or weak? When such studies are reviewed, a meta-analysis is probably even more problematic than a narrative review, because biased or incomparable risk estimates are formally forced into a single meta-risk estimate, which then becomes meta-biased. Hence, when researchers ponder whether or not they should initiate a project, especially if the literature is inconclusive, the availability of accurate and detailed exposure data is crucial. If good exposure data cannot be acquired, such a project will lead nowhere and should not be carried out at all. In some countries the cancer registers needed for good epidemiology do not exist, or their quality is low. In other countries data protection legislation hinders the use of registers for research. However, in these very same countries, exposure levels may still be high enough to cause measurable cancer risks, and the temptation to start a research project may be irresistible, although informative epidemiologic research may be difficult or impossible. Is it then worthwhile to start a project? Researchers' failure to realize that such, often unavoidable, limitations will render a study uninterpretable, even when carried out with technical skill, is one of the reasons why the literature becomes confused. One remedy would be for researchers and funding agencies to become much more critical when evaluating the prospects of a study to be informative. "Sample size zero" is surprisingly often the best study size. However, because the result of a study cannot be known with certainty in advance, the decision is not an easy one. Some chances have to be taken, but clear potential failures should be avoided. Scientific journals also share the responsibility of cleaning up the literature, but their possibility to act comes at a stage when much (unnecessary) work has already been done, and this is frustrating for everyone involved. As if these "natural" difficulties were not enough, researchers have, in addition, all too often violated good epidemiologic practice in matters that could and should have been controlled. When Kristensen prepared his classical reviews on work-related risk factors for cardiovascular disease, he first scored the articles into 5 categories according to quality (5). He found that a high score correlated closely with the magnitude of the risk estimate. In other words, the better studies were more sensitive in detecting an existing risk. The same is certainly true for cancer studies. A full list of the weaknesses that can be found in published studies due to lack of skill is too long to be recapitulated here. Let some examples suffice. In addition to neglecting the need for good exposure information, even if data would be available, important common errors include uncontrolled confounding, poor accuracy of the cancer diagnosis, use of referents who are not representative of the study base (such as the general population in cohort studies and dubious secondary study bases in case-referent studies), poor control of recall bias, poor control of selection, and many others. Especially the use of the general population as a reference category has created more confusion than most epidemiologists are willing to admit. (Why should they, because almost everyone has committed - and still commits - the sin of introducing this comparison bias, pseudoscientifically named the healthy-worker effect. My own record is not entirely clean, either.) The nested case-referent design, using a large study base with sufficient exposure contrasts (Note! Even unexposed people) overcomes the bias from the healthy-worker effect. Why not consider this alternative? Poor research should be avoided at all costs, and at least it should not be published. When it comes to publishing, so-called better journals function as gatekeepers, although with varying success. Their peer review process, together with a high rejection rate, helps improve the quality of what becomes published. However, all journals are not "better", and the wild growth of the number of periodicals guarantees that space is almost always available somewhere. Hence persistent authors will, in the end, find some journal willing to publish their mediocre study, and the literature becomes contaminated once again. Therefore, improving the quality of the research itself is more crucial than focusing on the rather hopeless task of completely preventing poor studies from being published. This approach is also more fruitful. Improving the quality of research is not only the responsibility of research institutions, but also of every individual researcher. Institutions must actively promote their researchers' skills. This is best done by providing positive incitements, such as giving financial support for training, creating an innovative atmosphere, and providing sufficient facilities. Courses in research methodology and other curricula should be organized - and attending them should be mandatory. Young researchers must also receive close attention from their superiors and be given personal guidance, even if it requires much time. Informal scientific discussions among colleagues, even if the topic is not directly related to the current project, are part of a fruitful learning process. Reading literature should be encouraged. Young researchers should first be incorporated into teams led by their seniors and not be given independent and difficult tasks too early in their career. Written guidelines for conducting good research can also help improve quality (6). In addition senior co-workers busy with administration and other duties should sometimes stop and ask themselves if their current multitude of tasks will, in the long run (or even sooner), cause them to lose touch with research. Hence continuous self-improvement is a must not only for younger researchers, but also for older ones. Besides - some people never become good researchers! They may become excellent in other fields, and sometimes a shift at an early stage in one's career is the best solution for everyone. Studies affected by serious errors should not be allowed to "create confusion" in reviews and meta-analyses. If the quality of the original studies is not well accounted for in reviews, and especially so in formal meta-analyses, it will be hard to rid the current literature of the confusion that is so typical. In other words, poor studies indeed should be neglected in reviews of all kinds. However, even more important than neglecting poor studies is producing good ones. Modern literature shows that improvement is underway. For example, many researchers have become aware of the importance of good exposure data, and methods for exposure assessment have shown much development. The same can be said of data analysis. In addition, the quality of published studies has improved. Thus, in spite of a ballast from the past (and to some extent also still from the present), the share of "good" studies in the aggregate is increasing. This is a promising trend, but not yet enough, because the challenges are becoming harder. Epidemiologic research is becoming more and more difficult as most major, easily detectable environmental and work-related cancer risks become identified. The focus is switching towards minor risks caused by weak carcinogens and low exposure levels, small etiologic fractions, exposure-response relationships, and interactions between the environment and genetic factors. Only good, that is, sensitive and valid, studies with enough statistical power can detect such factors. Concentrating research institutions' resources, in terms of funds, time and competent manpower, on relevant problems, selecting only study settings that can provide answers, and developing the skills of the researchers is what we now need for improving the quality of epidemiologic research and creating more comprehensible literature. When all this has succeeded, some time in the not too distant future, reviews and meta-analyses will hopefully reach more definite conclusions than those based on the literature available today. We may even be able to rule out environmental risk factors for some cancers.