Some Costs & Benefits of Cost-Benefit Analysis

Abstract The American administrative state has become a cost-benefit state, at least in the sense that prevailing executive orders require agencies to proceed only if the benefits justify the costs. Some people celebrate this development; others abhor it. For defenders of the cost-benefit state, the antonym of their ideal is, alternately, regulation based on dogmas, intuitions, pure expressivism, political preferences, or interest-group power. Seen most sympathetically, the focus on costs and benefits is a neo-Benthamite effort to attend to the real-world consequences of regulations, and it casts a pragmatic, skeptical light on modern objections to the administrative state, invoking public-choice theory and the supposedly self-serving decisions of unelected bureaucrats. The focus on costs and benefits is also a valuable effort to go beyond coarse arguments, from both the right and the left, that tend to ask this unhelpful question: “Which side are you on?” In the future, however, there will be much better ways, which we might consider neo-Millian, to identify those consequences: 1) by relying less on speculative ex ante projections and more on actual evaluations; 2) by focusing directly on welfare and not relying on imperfect proxies; and 3) by attending closely to distributional considerations–on who is helped and who is hurt.


Cass R. Sunstein
The American administrative state has become a cost-benefit state, at least in the sense that prevailing executive orders require agencies to proceed only if the benefits justify the costs. Some people celebrate this development; others abhor it. For defenders of the cost-benefit state, the antonym of their ideal is, alternately, regu lation based on dogmas, intuitions, pure expressivism, political preferences, or interestgroup power. Seen most sympathetically, the focus on costs and benefits is a neo-Benthamite effort to attend to the real-world consequences of regulations, and it casts a pragmatic, skeptical light on modern objections to the administrative state, invoking public-choice theory and the supposedly self-serving decisions of unelected bureaucrats. The focus on costs and benefits is also a valuable effort to go beyond coarse arguments, from both the right and the left, that tend to ask this unhelpful question: "Which side are you on?" In the future, however, there will be much better ways, which we might consider neo-Millian, to identify those consequences: 1) by relying less on speculative ex ante projections and more on actual evaluations; 2) by focusing directly on welfare and not relying on imperfect proxies; and 3) by attending closely to distributional considerations-on who is helped and who is hurt. F rom 1981 to the present, the American administrative state has become, to a significant extent, a cost-benefit state. 1 Under prevailing executive orders, agencies must calculate the costs and benefits of proposed and final regulations, and to the extent permitted by law, may proceed only if the benefits justify the costs. These requirements have spurred, and helped make possible, life-saving regulations in a variety of domains, including clean air, motor vehicle safety, clean water, homeland security, public health, climate change, and occupational safety. At the same time, they have served as a check on, and an obstacle to, regulations that would cost a great deal and achieve very little.
Of course it is true that political considerations matter, even in a cost-benefit state. In Congress, cost-benefit analysis often takes a back seat, if it makes it into the room at all. In the executive branch, political convictions, dogmas, or perceived electoral considerations may trump the outcome of cost-benefit analysis, or make it an ex post justification or an afterthought, rather than a driver of deci-sions. Nonetheless, the analysis of costs and benefits, offered by technical specialists, often has a real impact on regulatory choices, pressing administrators in the direction of greater or less stringency, exposing new options, or offering a bright green GO! or a forbidding red STOP! In terms of rigor, coverage, and accuracy, a great deal remains to be done. The fact that cost-benefit requirements do not apply to the "independent" agencies, such as the Federal Communications Commission, the Securities and Exchange Commission, and the Nuclear Regulatory Commission, is a continuing problem. Sometimes the numbers are based on guesswork, and there is continuing concern about whether before-the-fact estimates (of, for example, safety and health regulations) are reliable, or whether they are, on some occasions, a stab in the dark. Many people have argued for rigorous, ongoing evaluations, in which administrators test whether (for example) a regulation designed to increase food safety, or to protect against occupational injuries, is actually having its intended effect, and whether it is doing better or worse than expected. They are right to make that argument.
Despite the continuing challenges, the emergence of the cost-benefit state is a remarkable achievement. It means that the role of dogmas, intuitions, and interest groups has diminished and that within the executive branch, at least, regulators have often focused insistently on the human consequences of what they are proposing to do. To a significant extent, the cost-benefit state has been a check on "expressivism," in which public officials, on either the left or the right, act to express abstract values without exploring whether particular initiatives would actually have good or bad consequences. To the extent that the consequences of regulations are genuinely good (because, say, they prevent hundreds or thousands of deaths), the rise of the cost-benefit state casts a new light on some prominent and high-minded critiques of modern administration-for example, that it is a product of unelected bureaucrats, a tribute to the power of well-organized private groups, a reflection of monied interests, an unacceptable abdication of legislative authority, or a product of government's efforts to expand its own power.
To be sure, each of these critiques must be met on its own terms. But if (for example) a motor vehicle safety regulation from the Department of Transportation, authorized by Congress, is preventing three hundred deaths annually and costing just $40 million, it would not seem that there is good reason for complaint, and the same is true if the Environmental Protection Agency (EPA) is finding ways to reduce greenhouse gases significantly and at modest cost. Indeed, many regulations, under both Republican and Democratic administrations, have delivered massive net benefits (understood as benefits minus costs). It is not unusual to find that in a given year, the monetized benefits of regulations (including the benefits in terms of preventing illnesses, accidents, and premature deaths) exceed the monetized costs by many billions of dollars. (The Trump administration was an Some Costs & Benefits of Cost-Benefit Analysis outlier; because it issued so few regulations, the annual costs of what it did were very low, and so were the annual benefits.) Under favorable conditions, the use of cost-benefit analysis can provide safeguards against decisions based on feelings, hopes, presumptions, perceived political pressures, appealing but evidence-free compromises, broad aspirations, guesses, or the wishes of the strongest people in the room. But the administrative state should do better still. It needs to focus directly on human welfare. It should see cost-benefit analysis as a mere proxy for welfare, and an imperfect one to boot. It needs to investigate welfare itself, and to explore what that idea is best understood to mean. It needs as well to focus on distributional considerations-on who is helped and who is hurt.
To see the underlying problems, consider a realistic if highly stylized example. Suppose that the Environmental Protection Agency is considering a new regulation designed to reduce levels of particulate matter in the ambient air. Suppose that the total annual cost of the regulation would be $900 million. Suppose that the monetized mortality benefits would be higher than that-because, say, the regulation would prevent one hundred deaths, each valued at $10 million. (This is a hypothetical number; as of 2021, prominent federal agencies valued a statistical life at about $11 million.) Suppose as well that if the EPA includes morbidity benefits (in the form of nonfatal illnesses averted), the regulation would produce an additional $350 million in benefits, meaning that the monetized benefits ($1.35 billion) are significantly higher than the monetized costs ($900 million). At first glance, the cost-benefit analysis suggests that the regulation is an excellent idea, and that the EPA should go forward with it. Now assume four additional facts. First, the mortality benefits of the regulation would be enjoyed mostly by older people: those over the age of eighty. Second, the rule would have significant disemployment effects, imposing a statistical risk of job loss on a large number of people, and ultimately causing three thousand people to lose their jobs. Third, the EPA believes that the overwhelming majority of those three thousand people would find other jobs, and probably do so relatively soon, but it does not have a great deal of data on that question and it cannot rule out the possibility of long-term job loss for many people. Fourth, the mortality and morbidity benefits would be enjoyed disproportionately by low-income communities and by people of color. In accordance with standard practice, the EPA does not include any of those further facts in its cost-benefit analysis.
If the goal is to promote social welfare, it would be far too simple for the EPA to conclude that, because the monetized benefits exceed the monetized costs, it should proceed with the regulation. One question is whether and how to take into account, in welfare terms, the relatively few additional life-years that the regulation will generate. In those terms, is a rule that "saves" people over eighty to be deemed equivalent to one that "saves" an equivalent number of people who are 211 Cass R. Sunstein (say) under thirty? And what are the welfare consequences of the $900 million expenditure? Suppose that, concretely, the admittedly high cost will be spread across at least two hundred million people, who will be spending, on average, a little over $4 annually for the regulation. What are the welfare consequences of that modest expenditure? Might they be relatively small? (The answer is emphatically yes. Most people will lose essentially no welfare from an annual $4 loss.) A further question is the disemployment effect. We know that in terms of subjective welfare, it is extremely bad to lose one's job. 2 People who lose their jobs suffer a lot: Job loss can severely harm one's self-worth and experience of daily life. A sudden loss of income can threaten housing and food security, often causing disruptions to family life and schooling. A loss of a job also creates a nontrivial long-term loss in income. 3 If you are out of work for a year, the economic toll might be very high over a lifetime. We know that a long-term loss of employment has more severe adverse consequences than a short-term loss, but both are bad. Shouldn't those welfare effects be included?
Yet another question is the distributional impact. If the health benefits of regulation would be enjoyed mostly by members of low-income groups, and particularly by people of color, might that matter? We might think that even if the rule does not have significant net welfare benefits, or even if it has some net welfare costs, it is nonetheless desirable, if and because it increases equality. The interest in environmental justice focuses on the very real possibility that wealthy people might be the disproportionate beneficiaries of polluting activity and that poor people might bear most of the costs. (In the context of air pollution, that appears to be true.) These considerations suggest that while monetized costs and benefits tell us a great deal, they do not tell us everything that we need to know. On welfare grounds, a rule might not make sense even if the monetized benefits are higher than the monetized costs, and a rule might make sense even if the monetized costs are higher than the monetized benefits. In addition, we should want to consider distributional effects. To be sure, a rule that costs $1 billion and that provides benefits of $100 would not be a good idea even if the wealthy pay that $1 billion and poor people receive that $100. But if a rule costs $1 billion and delivers $950 million in benefits, we might want to go forward with it if the cost is diffused among a large number of wealthy people, and if the benefit is enjoyed by (for example) coal miners whose lives are at stake. Now suppose that the Department of Transportation is considering a regulation that would require all new automobiles to come equipped with cameras, so as to improve rear visibility and thus reduce the risk of backover crashes. 4 Suppose that the total estimated annual cost of the regulation is $1.2 billion (reflecting an average added cost of $300 per vehicle sold over the relevant time period). Suppose that the regulation is expected to prevent sixty deaths annually, for mone-tized annual savings of $540 million, as well as a number of nonfatal injuries and cases of property damage, for additional annual savings of $200 million. On the basis of these numbers, the Department is inclined to believe that the benefits of the rules are significantly lower than the costs.
At the same time, suppose that the Department is aware of four facts that it deems relevant, but that it is not at all sure how to handle. First, a majority of the deaths that the regulation would prevent would involve young children, between the ages of one and five. Second, a majority of those deaths would occur as a result of the driving errors of their own parents, who would therefore suffer unspeakable anguish. Third, the cost of the rule would be diffused across a large population of new car purchasers, who would not much notice the per-vehicle cost. Fourth, the cameras would improve people's driving experience by making it much easier for them to navigate the roads, even when it does not prevent crashes. (The Department speculates that many consumers do not sufficiently appreciate this improvement when deciding which cars to buy.) Is it so clear, in light of these four facts, that the agency should not proceed? That is not a hard question. The answer is: no. That answer suggests the importance of considering variables that are difficult or perhaps impossible to quantify. (How exactly to do that is a hard question.) In principle, cost-benefit analysis is best defended as the most administrable way of capturing the welfare effects of policies (including regulations). But if we actually knew those effects, in terms of people's actual welfare (suitably specified), and thus could specify the actual consequences of policies for welfare (again, suitably specified), we would not have to trouble ourselves with cost-benefit analysis. An initial problem is that cost-benefit analysis depends on willingness to pay, and people might be willing to pay for goods that do not have substantial positive effects on their welfare (and might be unwilling to pay for goods that would have substantial positive effects). Willingness to pay is based on a prediction, and at least some of the time, people make mistakes in forecasting how various outcomes will affect their lives (and make them feel). Call them welfare forecasting errors. You might think that if you do not get a particular job, or if your favorite sport team loses a crucial game, or even if someone you really like refuses to date you, you will be miserable for a good long time. But chances are that you are wrong; you will recover much faster than you think. The basic point applies to the administrative state and its choices. People might make welfare forecasts with respect to calorie consumption or exposure to certain risks, and those forecasts might go wrong. If administrators rely on welfare forecasts as reflected in willingness to pay, they might incorporate and hence propagate errors.
A separate problem involves the incidence of costs and benefits, which can complicate the analysis of welfare effects, even if we put "pure" distributional considerations to one side. Suppose that a regulation would impose $400 million in costs on relatively wealthy people and confer $300 million in benefits on relatively poor Cass R. Sunstein people. Even if the losers lose more than the gainers gain in monetary terms, we cannot exclude the possibility that the losers will lose less than the gainers gain in welfare terms.
An additional problem is that because willingness to pay depends on ability to pay, it can be a poor measure of welfare effects. A very rich person might be willing to pay a lot (say, $2,000) for a good from which she would not get a lot of welfare. (After all, losing $2,000 is a trivial matter, if you are very rich.) A very poor person might be willing to pay only a little (say, $20 and no more) for a good from which she would get a lot of welfare. (After all, losing $20 is no trivial matter, if you are very poor.) These points do not mean that a very rich person should be prevented from paying that large amount for that good, or that a very poor person should be forced to pay more than that small amount for that good. (People who like regulation often miss the latter point in particular.) But they emphatically do mean that if a very poor person, or simply a poor person, is willing to pay only a small amount to avoid a mortality risk, or to get some benefit (say, an unlawfully present citizen seeking "deferred action" from the U.S. government), that small amount is not a good measure of the welfare effects.
The most general problem is that whenever agencies specify costs and benefits, the resulting figures will inevitably have an ambiguous relationship with what they should care about, which is welfare. To be sure, it is possible that some of the problems in the two cases I have given could be significantly reduced with improved cost-benefit analysis. If children should be valued differently from adults, and elderly people differently from younger, cost-benefit analysis might be able to explain why and how. Perhaps parental anguish could be monetized as well. (Why, you might ask? It is a fair question. The answer is to figure out how to weigh both sides of the ledger; without that, how can a regulator make a sensible decision?) The same might well be true, and might more readily be true, of the increased ease of driving. But even the best proxies remain proxies, and what matters most is welfare itself. I n recent years, social scientists have become greatly interested in measuring welfare. One of their techniques is to study "self-reported well-being," meaning people's answers to survey questions about how satisfied they are with their lives. The promise of this technique is that it might be able to offer a more direct, and more accurate, measure of welfare than could possibly come from an account of costs and benefits (especially if that account depends on willingness to pay). 5 Suppose that we agree with economist Paul Dolan that welfare largely consists of two things: 1) people's feelings of pleasure (broadly conceived) and 2) people's feelings of purpose (also broadly conceived). 6 People might enjoy watching sports on television, but they might not gain much of a sense of purpose from that activity. Working for a good cause (consider working for a nonprofit or for a government whose leaders you admire) might not be a lot of fun, but it might produce a strong sense of purpose.
If pleasure and purpose matter, and if we want to measure them, we might be able to ask people about those two variables. How much pleasure do people get from certain activities? How much of a sense of purpose? Dolan has in fact asked such questions, with illuminating results. 7 We are learning a great deal about what kinds of activities are pleasurable or not, and also about what kinds of activities seem to give people a sense of purpose or meaning. In the abstract, what we learn seems to tell us a lot about people's welfare, and it might offer a more direct and accurate account than what emerges from an analysis of costs and benefits. The reason is that measures of pleasure and purpose offer information about people's actual experience of their lives, rather than a projection as measured by money, and the former seems to be what most matters.
With respect to subjective well-being, the most popular existing measures take two forms. First, researchers try to assess people's "evaluative" welfare by asking questions about overall life satisfaction (or related concepts, such as happiness). 8 With such measures, it is possible to test the positive or negative effects of a number of life events such as marriage, divorce, disability, and unemployment. 9 Second, researchers try to assess people's "experienced" welfare, through measures of people's assessments of particular activities (working, commuting, being with friends, watching television). 10 In fact, researchers have uncovered some systematic differences between people's overall evaluations and their assessments of their particular experiences. 11 Marital status is more closely correlated with experienced well-being than with evaluative well-being, though there is conflicting evidence on this point. 12 French people report significantly lower levels of satisfaction with their lives than Americans, but the French appear to show equal or even higher levels of experienced well-being. 13 (Psychologist Daniel Kahneman has suggested a partial explanation: in France, if you say you are happy, you are superficial; in the United States, if you say you are unhappy, you are pathetic.) Health states are more closely correlated with experienced well-being, though they also affect evaluative well-being.
How can the choice be made between the two measures? 14 The emerging consensus is that useful but different information is provided by each. On one view, questions about experienced welfare focus people on their existing emotional states, and thus provide valuable information about those states. By contrast, questions about evaluative welfare encourage people to think about their overall goals or aspirations. On this view, evaluative welfare "is more likely to reflect people's longer-term outlook about their lives as a whole." 15 If this is so, then the two measures do capture different kinds of values, and both are important. But it is not clear that the emerging consensus is correct, for a critical question remains: do people's answers to questions about evaluative well-being in fact reflect their broader aspirations, or do they represent an effort to summarize experienced well-being (in which case the latter is the more accurate measure)?
True, the idea of "welfare" leaves a great deal of ambiguity, and if it is invoked for policy purposes or by governments, any particular account is highly likely to end up in contested terrain. 16 As made clear by Dolan (not to mention Aristotle, John Stuart Mill, and Amartya Sen), a neo-Benthamite measure, purely hedonic and focused only on pleasure and pain, would be inadequate; people's lives should be meaningful as well as pleasant. But even if we adopt a measure that goes beyond pleasure to measure a sense of purpose as well, we might be capturing too little. We might be ignoring qualitative differences among goods and the general problem of incommensurability.
We value some things purely or principally for use; consider hammers, forks, or money. We value other things at least in part for their own sake; consider knowledge or friendship. But that distinction captures only part of the picture. Intrinsically valued things produce a range of diverse responses. Some bring about wonder and awe; consider a mountain or a work of art. Toward some people, we feel respect; toward others, affection; toward others, love. (There are of course qualitative differences among different kinds of love.) Some events produce gratitude; others produce joy; others are thrilling; others produce a sense of wonder; others make us feel content; others bring about delight. Some things are valued if they meet certain standards, like a musical or athletic performance, or perhaps a pun. In this regard, Mill's objections to Bentham are worth quoting at length: Nor is it only the moral part of man's nature, in the strict sense of the term-the desire of perfection, or the feeling of an approving or of an accusing conscience-that he overlooks; he but faintly recognizes, as a fact in human nature, the pursuit of any other ideal end for its own sake. The sense of honour, and personal dignity-that feeling of personal exaltation and degradation which acts independently of other people's opinion, or even in defiance of it; the love of beauty, the passion of the artist; the love of order, of congruity, of consistency in all things, and conformity to their end; the love of power, not in the limited form of power over other human beings, but abstract power, the power of making our volitions effectual; the love of action, the thirst for movement and activity, a principle scarcely of less influence in human life than its opposite, the love of ease. . . . Man, that most complex being, is a very simple one in his eyes. 17 These points suggest the importance of having a capacious conception of welfare, one that is alert to the diverse array of goods that matter to people. Consistent with Mill's plea, a large survey by the economist Daniel Benjamin and coauthors tests people's concern for a list of factors that includes not only "measures widely used by economists (e.g., happiness and life satisfaction)," but also "other items, such as goals and achievements, freedoms, engagement, morality, selfexpression, relationships, and the well-being of others. The central and important (though not especially surprising) result, compatible with Mill's point, is that people do indeed care about those other items. 19 The perhaps ironic conclusion is that, if measures of reported well-being neglect those items, they will end up losing important information that cost-benefit measures ought to be able to capture. A significant advantage of the willingness-to-pay measure is that it should, in principle, take account of everything that people care about, including those things that matter for Mill's reasons. If people value cell phones because they want to connect with their children, or if they want to save (rather than spend) money so they can give it to impoverished children, or if they want to spend money on a vacation because of their love of nature, their concerns, however diverse in qualitative terms, should be adequately captured by the willingness-to-pay criterion, however unitary.
That is a point for cost-benefit analysis. Notwithstanding its apparent crudeness, and notwithstanding the simplicity of the monetary measure, it honors qualitatively diverse goods that people care about for diverse reasons. In that way, it is not simple at all, and for that reason, cost-benefit analysis has advantages over some measures of happiness or subjective welfare. Nonetheless, that form of analysis cannot have priority over excellent or full measures of welfare. What is required are measures that are sufficiently reflective of the diverse set of goods that matter to people but that avoid the various problems, signaled above, of costbenefit analysis. W ith respect to regulatory policy, the largest problem with invoking self-reported well-being is this: even if such surveys provide a great deal of information, we cannot easily "map" any particular set of regulatory consequences onto changes in welfare.
Although we are learning a great deal about what increases and what decreases welfare, what we are learning is relatively coarse; it frequently involves the consequences of large life events, such as marriage, divorce, and unemployment. 20 We do not know nearly enough about how to answer hard questions about the welfare effects of health, safety, and other regulations. For example: 1) How much happier are people when the level of ozone in the ambient air is decreased from seventy parts per billion to sixty parts per billion? 2) For the median person, what is the welfare effect of having to spend $50 or $100 or $300 on a particular regulatory initiative, noting that the money could have been used for other purposes? 3) What are the welfare effects of giving unlawful noncitizens in the United States deferred action, meaning that they will not be deported and will be authorized to work? 4) In terms of "welfare units," how should we think about a loss of a job, or a life-year? Should we use those units or some other kind of unit (monetary?) in conducting analyses on the basis of studies of self-reported well-being? If we use welfare units, what, exactly, is the relevant scale? Cass R. Sunstein Return to the two problems with which I began. We have seen that in terms of welfare, cost-benefit analysis, at least in its current form, may not adequately handle: 1) unusually large or unusually small numbers of life-years saved; 2) adverse unemployment effects; 3) questions about the welfare effects of small economic losses faced by large populations; 4) intense emotions associated with certain outcomes, such as parental anguish (or fear); and 5) hedonic benefits associated with increased ease and convenience. We have also seen that cost-benefit analysis does not capture distributional impacts, and that they might greatly matter. As I have suggested, improved forms of cost-benefit analysis might be able to reduce these problems (and cost-benefit analysis can of course be complemented with other inquiries; we might engage in that form of analysis and deal with distributional impacts separately). But ideally, we would want to know about welfare itself. The problem is that measures of self-reported well-being are far too crude to enable us to do that.
No one should doubt that cost-benefit analysis itself presents serious challenges, sometimes described under the rubric of "the knowledge problem": agencies have to compile a great deal of information to make sensible extrapolations. But to map regulatory outcomes onto self-reported well-being, the challenges are far more severe. Does this conclusion mean that today and in the near future, regulators should rest content with cost-benefit analysis, and put entirely to one side, as speculative and unreliable, whatever we might learn from directly considering welfare? That would be too strong. Most important, disemployment effects deserve serious consideration, not least because of the significant adverse welfare effects of losing one's job. It is also relevant to know whether a regulation would protect children, and hence provide a large number of life-years, or instead (and this is a far more controversial question) protect older people, and hence provide a relatively smaller number of life-years. The Department of Transportation was correct to emphasize that its rear visibility rule would disproportionately protect children.
It is also possible that a large cost, spread over a very large population, might turn out to have relatively modest adverse effects on welfare. Agencies should consider this possibility, especially in cases in which costs and benefits are otherwise fairly close. And if agencies would (for example) help people who suffer from mental illness of one or another kind, the welfare gain might be substantial, even if the benefits cannot be adequately captured in willingness-to-pay figures. Distributional effects should also be considered; they matter.
Emphasizing the promise of research on subjective well-being, economist Raj Chetty contends: "Further work is needed to determine whether and how subjective well-being metrics can be used to reliably measure experienced utility, but they appear to offer at least some qualitative information on ex post preferences [that] can help mitigate concerns about paternalism in behavioral welfare economics." 21