Risk Analysis and Cost-Benefit Analysis of Environmental Regulations

94-961 ENR

TABLE OF CONTENTS FOR THIS SECTION

[ANALYSIS
-- ISSUES IN ENVIRONMENTAL RISK MANAGEMENT]
-- -- Should Priorities Be Based on Relative Risks?
-- -- Would Additional Analysis Improve Risk Management?
-- THE INFORMATION VALUE OF RISK ANALYSIS
-- -- Key Factors Determining the Quality of Information Provided
-- -- Quality of the Database
-- -- Risk Assessment Methods
Some Subjective Judgements in Risk Assessment

Should Priorities Be Based on Relative Risks?

EPA's experiments with comparative risk analysis, and similar State experiments that were encouraged and funded by EPA, have generated considerable interest among legislators. Many would like scientists to provide information on how environmental hazards rank based on risk estimates, believing that this information would facilitate decisions about legislative and regulatory priorities. Some have suggested that Federal, State, and local governments should enact budgets and allocate resources that tie the greatest expenditures to environmental hazards posing the greatest risks. Others have proposed that EPA provide perspective for viewing environmental hazards within the context of other Federal programs; they would require EPA to compare the risks of regulated environmental hazards with risks of other regulated and unregulated hazards.

Those who object to comparative risk proposals contend that comparative risk analysis is an unscientific, ad hoc procedure that lends a false air of objectivity to the subjective judgments of scientists. They question whether an exercise that combines the diverse views of an unrepresentative sample of Government scientists to produce a single prioritized list of hazards is more informative than a thorough recitation of the points on which scientists with diverse viewpoints agree and disagree, such as may occur in a hearing or an advisory committee. Priority setting requires value judgments, they argue, and should be made politically; scientists are no more qualified than others to decide whether, for example, the risk of a small decrement in intelligence for 3 to 4 million children exposed to lead-based paint is more or less significant than the risk of approximately 13,600 deaths annually from lung cancer due to indoor levels of radon gas. Which is worse, one person dying or 10,000 people feeling sick most of the time? Does it matter if the one dying is a child, or a smoker? It is even more difficult and less scientific to compare ecological risks with risks to human health, these critics contend. According to this view, scientists are expert only at determining probabilities, and the public or its representatives should be asked to contribute their expertise to the process of priority setting. EPA's Science Advisory Board agrees with this view. In Reducing Risk it stated:

... because they experience those risks first-hand, the public should have a substantial voice in establishing risk-reduction priorities. Thus EPA should include broad public participation in its efforts to rank environmental risks. Such participation will help educate the public about the technical aspects of environmental risks, and it will help educate the government about the subjective values that the public attaches to such risks. The result should be broader national support for risk-reduction policies that necessarily must be predicated on imperfect and evolving scientific understanding and subjective public opinion.(52)

Critics of risk management based on risk comparisons alone (that is, relative risks) also argue that risk is only one aspect of the environmental problems. Some argue that risk comparisons often focus on average national rates of death or disease and ignore equally important factors, such as the acceptability of available risk reduction strategies or the fairness of the result. Priorities should be based on all relevant information about hazards and available management options, not on risk alone, and should be made democratically, they assert. They reason that all means of risk reduction are not equally desirable, citing diverse examples such as the wearing of a gas mask or modification of a production process to reduce use of toxic chemicals.(53) Benefits provided by hazards also vary and should be considered, according to this view, because risk is not always undesirable, and many risks, such as driving a car or skydiving, are taken voluntarily either for the benefits that may be obtained or for the thrill of the experience. Panelists at the CRS workshop agreed that risk analysis, especially quantitative assessment, tends to emphasize the magnitude and severity of consequences over other aspects of the situation, such as whether exposure to the hazard is necessary or voluntary or whether the people who profit from a hazard are the same as the people who are at risk or the people who pay to reduce risk.

In 1989, the NAS published a report that summarized the state of knowledge about how best to communicate about risks.(54) This report is noteworthy primarily because it was a consensus document prepared by the Committee on Risk Perception and Communication of the National Research Council (NRC), a large group of scholars and practitioners with widely differing political perspectives. One topic addressed by the report was how to employ risk comparisons. According to the Committee, risks should only be compared when risks are comparable; risks are comparable when they "exhibit qualitative characteristics that are reasonably similar."(55) This means that the most comparable risks generally are experienced in the same way by the same population for the same reason. For example, the risks of riding a bicycle, walking, or being driven to school may be easily compared. In contrast, the risk of being struck by lightening is not comparable to the risk of traveling, because although both are familiar, the former derives from a natural phenomenon beyond human control; in addition, an individual's risk of being killed by lightening is greatly reduced with little effort, for example by remaining indoors during storms or employing devices to deflect electrical charges. Several qualitative characteristics of hazards (risks) are particularly important to comparability, according to experts including: the magnitude and severity of the potential harm, likelihood of harm, voluntariness of exposure, immediacy of effect, trustworthiness of people managing the hazard (risk), population likely to be exposed, concentration of effects in time and space, population likely to benefit from the activity that creates the risk, and the degree of familiarity of (or adaptation to) the hazard. In addition, the NRC committee cautioned against appearing to select risks for comparison that "minimize or otherwise trivialize the risk in question," for example, by comparing a hazard (risk) like lightening that seems highly unlikely to inflict personal harm to one that is less well understood by scientists and causes deep distress to some individuals, like hazardous waste sites.(56)

Many proponents as well as opponents of risk comparisons agree that if hazards are compared based on risk estimates, risks to subpopulations should be considered as well as risks to the population as a whole. In addition, some propose that distinguishing characteristics of hazards that may affect the acceptability of risks should be highlighted. Committees of jurisdiction reported several bills in the 103rd Congress that required risk comparisons as well as identification of distinguishing characteristics of hazards and consideration of risks to vulnerable subpopulations.

Would Additional Analysis Improve Risk Management?

Proponents of risk analysis, comparative risk analysis, or economic analysis of the potential effects of environmental regulations have various options for promoting the activity. This section examines selected potential impacts of seven general approaches: 1) authorizing or requiring EPA to analyze regulations, 2) authorizing or requiring EPA to consider costs, relative risks, or the relationship between costs and benefits in making regulatory decisions, 3) requiring EPA to report to Congress on the results of regulatory analysis, 4) authorizing additional funding for analysis, 5) mandating research and development of analytic methods, the database, or guidelines for risk assessment, 6) establishing guidelines for risk assessment or the presentation of analytic results, or 7) requiring peer review.

In general, provision of statutory authority to conduct or to consider the results of analyses probably would generate less controversy than a mandate, which the Executive might oppose. Some argue that constraints on the prerogatives of managers may violate tenets of good management by reducing the flexibility needed to allocate resources efficiently and to adapt quickly to changing circumstances.

The potential effect on EPA of legislation authorizing or requiring economic or risk analysis or consideration of risks, relative risks, costs, and/or benefits in developing regulations would depend largely on which regulations were to be analyzed. Legislation affecting only "major" or ''significant'' regulations might have relatively less impact, because EPA already is required, to the extent permitted by law, to conduct and consider the results of cost-benefit-risk analysis for all regulatory actions that are "significant." Statutory requirements would apply even in the absence of an executive order.

Legislation requiring analysis of additional regulations could provide information that now is not readily available. Information about alternative regulatory strategies and their potential consequences (that is, implementation and compliance costs, the risks avoided, and other benefits) could help policymakers and the general public set priorities, allocate resources, and evaluate existing Federal laws and programs.

On the other hand, additional requirements for analysis would require additional resources. The Congressional Budget Office has estimated that it would cost $20 million to analyze all "non-routine" EPA regulatory actions (approximately 50 percent of all regulatory actions.) In the absence of additional resources, the increased number of required analyses might force EPA to reduce the quality of analysis for "significant" regulations. If EPA were to sacrifice analytic quality underpinning its regulations, it might be more vulnerable to legal challenges from regulated entities.

Additional private expenditures also might result from new statutory requirements, because EPA's analyses often use data provided by regulated industries. To supply data for additional analyses, EPA might require additional data collection and reporting by such industries.

Additional requirements for analysis might delay EPA implementing provisions of major environmental statutes. Any such delays would likely anger environmental groups, who might seek to judicially compel EPA action. Delays could also affect regulated industries, for example, if the regulations delayed were meant to clarify or reduce existing regulatory requirements.

Such delays or increases in the cost of regulating might be reduced, however, if legislation authorized analyses that varied in detail in proportion to the significance of regulations. CRS workshop panelists believed that Federal agencies sometimes analyze more than is necessary to inform risk managers. To the extent that this is true, EPA might be able to reduce the overall cost of promulgating regulations, if Congress gave it the authority to vary the depth of analyses.

A Federal mandate to conduct risk analysis to support regulations also might affect data collection, according to CRS workshop panelists. They cautioned that a mandate might discourage industries from doing research and collecting data, because once data are produced, for example, on chemical toxicity, EPA is perceived as being more likely to conduct risk analysis and to regulate.

Requiring analysis or consideration of risks and costs would not necessarily prevent regulation of very small risks. Nor would such a requirement prevent promulgation of regulations that are costly for regulated industries or State or local governments. Since 1981, EPA has almost invariably conducted risk and economic analyses in developing its more costly regulations. Critics have claimed that in come cases the costs appear high relative to the risks they address. Many of EPA's critics believe the Agency is simply too protective of health and the environment. However, an equally plausible explanation for such regulations may be that authorizing statutes constrain EPA (or are interpreted by EPA to constrain it) from considering costs or the magnitude of risk when setting the standards or safety criteria.

If it is determined that some statutes require EPA to regulate insignificant risks regardless of cost, Congress could choose to override existing statutory authority by omitting the standard "saving' clause from legislation that requires consideration of risks, costs, or cost-effectiveness of regulations. However, this would likely require review of each potentially affected environmental statute: an overriding statute might have unintended consequences that would be difficult to predict. In addition, significant opposition might be expected to such legislation because it might appear to reduce the overall level of protection of the environment and public health. Alternatively, Congress might consider amending the requirements for analysis in each environmental statute.

Instead of authorizing or requiring EPA analysis or consideration of risks, costs, or relative risks, legislation could require periodic EPA reports on the results of such analyses of environmental regulations; such reports might assist Members with oversight responsibilities or alert Members on authorizing committees to provisions in authorizing statutes in need of reexamination. Such reports arguably would consume relatively less of EPA's resources and might serve additional purposes, for example, to inform the general public about Federal programs.

Given the historically high level of EPA involvement in risk and economic analyses and the fact that EPA has conducted analyses of proposed regulation for two decades, often on its own initiative, Congress might conclude that inadequate funds help explain perceived regulatory failures. In this case, Congress might choose to provide additional funding targeted to EPA's analytic activities; more and better analyses and better regulatory decisions might be obtained. A similar argument might be made for eliminating statutory deadlines contained in authorizing statutes: by providing additional time for analysis prior to promulgation of regulations, Congress might enable more comprehensive analyses and more rational decision processes.

Because the NAS has concluded that the greatest improvement in risk analysis might be obtained by improving the quality and comprehensiveness of knowledge, Congress might choose to authorize or mandate EPA attention to research and development.(57) Analytic methods, data collection, or guidelines might be targeted for development. Such legislation would likely generate less controversy.

Finally, those who believe that EPA is dominated by political rather than scientific considerations might prefer to impose scientific guidelines or standards for risk assessment or the presentation of analytic results. Such guidelines or standards could be mandated by Congress, to be developed by EPA, another agency, an interagency workgroup, or an outside body with the relevant expertise. It is questionable, however, whether any group could develop detailed guidance that would be applicable to the array of environmental problems and accepted as "scientific" and unbiased. Many groups have tried to accomplish this task and failed over the last 25 years. In addition, detailed guidance may be difficult to update quickly enough to keep up with rapid changes in science.

Alternatively, Congress could encourage or require development of a system for independent peer review of risk analyses and economic analyses that underpin proposed regulations. Peer review is a familiar and well-established practice among scientists and it generates little controversy. Scientists have found no other means to be as effective for enforcing high standards of quality for scientific publications. Only peer review has been found to be flexible enough to respond quickly to changes in scientific knowledge and methods.

THE INFORMATION VALUE OF RISK ANALYSIS

There appears to be general agreement that policymakers need more information to inform risk management decisions. Views diverge, however, regarding the type of information needed and whether it is best provided by risk analysis. The debate might benefit, therefore, from explicit consideration of what information risk analysis provides.

Key Factors Determining the Quality of Information Provided

Under ideal conditions, a risk analysis might gather, organize, and summarize all of the important information relevant to hazard management. It would include qualitative as well as quantitative information about the characteristics of the hazard, exposed population, potential effects, and potential effects of available management strategies; describe scientific uncertainties; and provide a range of forecasts based on alternative, scientifically plausible assumptions about the relationship between exposure to the hazard and potential health or environmental effects.

In practice, however, the type of information provided by environmental risk analysis varies from abundant (but often with critical gaps) to superficial, from accurate to biased, because risk analysis is a field of inquiry rather than a single method. Risk analysts study hazards using a variety of procedures adapted from other fields of study. Sanitary and industrial engineering, psychology, economics, sociology, statistics, and operations research, for example, have provided models and procedures used by risk analysts. Because some of these methods were developed for different purposes (for example, to determine actuarially sound life insurance rates), they often have not been scientifically validated for, and are difficult to apply to, environmental hazards. The defining characteristic of methods used in risk analysis is a reliance on past experience to predict future events. If there is no past experience, there are no data and there can be no meaningful analysis.

A second consideration is that risk analysis is a tool for evaluating what is known about things that cannot be known with certainty -- that is, it is only used to describe the effects of hazards that are unpredictable due either to their randomness or to lack of data or scientific understanding of the principles that govern their occurrence. Its methods were developed to allow agencies to implement legislation despite incomplete data and scientific understanding.

Risk analysis always produces an estimate, never an exact prediction, and estimates vary in quality. (Weather forecasts, for example, are relatively well-informed risk estimates.) Thus, risk analysts can only discuss the likelihood of various outcomes and, at best, may present risks as statistical probabilities. If there is no past experience with a hazard, there is no basis for any forecast, much less a quantitative estimate (although risk estimates may be made based on conceptual models or experiences with similar hazards.) If there is experience but no record to ensure accurate recall, risk estimates are likely to be unreliable.

Finally, there are times when risk analysis can provide no information at all, because some environmental hazards and effects defy risk analysis, even when data are abundant. Science cannot always explain complex or unusual relationships between the exposures to hazards and the potential health and ecological effects. For example, chemicals in the environment that suppress immune systems may not be recognized as hazards, because their effects will be seen as a variety of health problems, each of which may be attributed to a different cause. In other cases, only people with certain innate characteristics may be affected by exposure to a toxic substance.

Quality of the Database

The quality of available data largely determines the quality of information that can be provided by a risk analysis. Thus, the NAS concluded in 1983 that the most effective way to improve risk assessment in the Federal Government is to improve the quality and comprehensiveness of knowledge.(58) The current data situation was summarized in a recent report by the Congressional Office of Technology Assessment (OTA).(59) It estimated that 62,512 chemicals are in commerce in the United States today, and another 1500 new chemicals enter the market annually. Environmental experts believe that "good" data on health effects exist for only 10% of commercial chemicals, according to OTA. Of course, many of these new chemicals that have not been tested adequately may be harmless, but according to NAS, data are also inadequate for many chemicals that Congress has deemed "hazardous." NAS recently evaluated the availability of data for risk analyses for 189 hazardous air pollutants and concluded EPA did not have "sufficient data to assess fully the health risks ... within the time permitted by the Clean Air Act Amendments of 1990.(60) OTA reported that at least 12 Federal agencies are currently conducting health risk assessment research to fill the gaps in scientific understanding, but their efforts are poorly coordinated and supported at a level that is less than 0.5 percent of the cost of complying with EPA regulations. (This figure does not include research relevant to the analysis of ecological risks.)

Risk Assessment Methods

Environmental risk analysis is a relatively new and immature field, and this is evident in the state of development of its methods. The most developed and well established analytic methods probably are those concerning acute effects; for chronic effects, the most developed are those used to assess human cancer risks of chemicals. These methods evaluate and model the results of animal experiments and human studies to estimate the risk that people will develop cancer following various levels of exposure to individual chemicals. Many of EPA's environmental standards, emission limits, and quality criteria are based on the results of cancer risk assessment. Other categories of risks, such as mutagenicity and immunotoxicity are rarely assessed, representing a substantial hole in risk assessment methodology.

Even cancer risk assessment is beset by the absence of scientific data and theories. The scientific judgments and inference choices that are used to fill these gaps are controversial, because they are shaped by a scientist's values, different scientists have different values, and different choices lead to different risk estimates. Many social scientists who study technical controversies believe that "ostensible disputes over the science are, in reality, over the values inherent in the assumptions."(61) The NAS has identified at least 50 inference choices required in conducting a cancer risk assessment that cannot be made on a scientific basis, and many of these decisions have strong implications for public policy. For example, analysts must determine how much evidence is enough to conclude that a chemical is a possible human carcinogen. Some want strong evidence prior to classification; they prefer to err, if necessary, by withholding judgment until they are sure a problem exists. Others would act on the first available evidence; they prefer to err on the side of alerting public officials to a possible risk.

To reduce the influence of values on individual risk estimates and to ensure that the assumptions and inferences choices made by agencies are clearly expressed, NAS has suggested that Federal agencies should develop and adopt guidelines for risk assessment. EPA adopted the first guidelines for cancer risk assessment in 1977. A revision was promulgated in 1986, and a second revision is in progress.

EPA also established guidelines in 1986 for analyzing: the risk that a chemical will cause mutations affecting future generations or damage to human development; human exposure to a chemical; and human health risks of chemical mixtures. The Agency revised its guidelines for developmental toxicants in 1991 and for exposure in 1992. EPA's cancer risk assessment guidelines currently are being revised. In addition, guidelines are being developed for analyzing neurotoxicity and reproductive and for exposure measurements. The Agency has proposed a rough framework for ecological risk analysis based on recommendations of the NAS.(62) No guidelines are established for assessing the risk of a chemical's adverse effects on the nervous, respiratory, or immune systems, or for other lethal and sublethal effects. There is no established scientific procedure for assessing ecological risks or for conducting comparative risk analysis.

Guidelines are necessary to ensure that risk assessments are conducted consistently and, therefore, are more easily evaluated by independent experts. (Independent evaluations of risk assessment by qualified experts, or peer review, is the traditional means by which scientists ensure adherence to professional standards of quality in practice.) However, guidelines do not ensure that equally competent scientists will agree with the risk estimates produced by the process. In fact, some scientists criticize EPA's risk estimates for carcinogens because they do not agree with the guidelines or think different rules should apply to certain chemicals. Controversy surrounds risk assessment only partly because the field is so young that its methods have not been studied thoroughly and adequately validated. NAS concluded in its 1983 report:

Dissatisfaction with the actions of [F]ederal regulatory agencies is often expressed as criticism of the conduct and administration of the risk assessment process. The Committee believes that the basic problem in risk assessment is the sparseness and uncertainty of the scientific knowledge of the health hazards addressed, and this problem has no ready solution. The field has been developing rapidly, and the greatest improvements in risk assessment result from the acquisition of more and better data, which decreases the need to rely on inference and informed judgment to bridge gaps in knowledge" (p. 5-6).(63)

Thus, controversy will not disappear when risk analysis matures, because it grows inevitably from value judgments based on different ethical systems and inference choices embodied in agencies' science policies which make risk assessment possible as well as from the special interests that stakeholders have in EPA's risk estimates.

SOME SUBJECTIVE JUDGMENTS IN RISK ASSESSMENT

• What kinds of evidence are needed to demonstrate carcinogenicity?
• How important are toxicity studies that show an effect relative to studies that show no effect?
• How are benign and malignant tumors in animals counted?
• What are the appropriate dose levels for experiments?
• How should animal doses be compared to human doses?
• How should animal effects be compared to human effects?
• Are the effects observed at high doses expected to occur at low doses?
• Should different chemical carcinogens be treated differently?
• How should carcinogenicity be compared to mutagenicity? To birth defects?
• Source: Adapted from Rushefsky, M. Making Cancer Policy. Albany, NY, State University of New York Press, 1986. p. 40.

Endnotes

52 Ups. EPA, Science Advisory Board. Reducing Risk: Setting Priorities and Strategies for Environmental Protection. Washington, U.S. Environmental Protection Agency. (1990) p. 24.

53 In general, risk may be eliminated, avoided, or reduced by eliminating, controlling, or isolating the hazard. Alternatively, risk may be reduced without affecting the hazard by preventing or reducing exposure of people or other living things to the hazard. A population also might be compensated for any adverse effects experienced in the event of exposure. The risks of specific hazards may or may not be easily controlled in any of these ways.

54 National Research Council. Improving Risk Communication. Washington, D.C., National Academy Press, 1989. p. 97.

55 Ibid.

56 Ibid.

57 NAS, National Research Council. Risk Assessment in the Federal Government: Managing the Process. Washington, DC, National Academy Press, 1983. p.5-6.

58 NAS, National Research Council. Risk Assessment in the Federal Government: Managing the Process. Washington, DC, National Academy Press, 1983. p.5-6.

59 U.S. Congress, Office of Technology Assessment. Researching Health Risks, OTA-BBS-570. Washington, U.S. Govt. Print. Off., Nov. 1993. 928 p.

60 NAS, National Research Council. Science and Judgment in Risk Assessment. Washington, National Academy Press. (1994) p. 8-13.

61 Rushefsky, Mark E. "Assuming the Conclusions: Risk Assessment in the Development of Cancer Policy." Politics and the Life Sciences, v. 4, (August), 1986. p. 31.

62 U.S. EPA, Risk Assessment Forum. Framework forEcologicalRiskAssessment.EPA!630/R-92/001. Washington, U.S. Environmental Protection Agency, February 1992. 41 p.

U.S. EPA, Risk Assessment Forum. Report on the Ecological Risk Assessment Guidelines Strategic Planning Workshop. EPA/630/R-92/002. Washington, U.S. Environmental Protection Agency, February 1992. 57 p.

U.S. EPA, Risk Assessment Forum. Draft Ecological Risk Assessment: Issue Papers. EPA/630/R-94/004A. September 1993. 544 p.

63 Ibid.