Lecture 7: Fallacies & Risks

Four Points About Arguments

1. Building your case on mere possibility

-In general, a weak strategy

-Anything’s possible

-Reasonable possibility is generally much more interesting

-Remember that when arguing about which engineering outcome to choose, all options will typically carry some possibility of a catastrophic outcome

Fallacies

A fallacy is a common error in reasoning which people (even well educated, careful people) often fail to notice in their own arguments or which devious people might use in their arguments in the hope that we won't notice them.

Four Points About Arguments

2. Mistaking Possibility for Probability

-E.g., there is a chance that containment will fail, therefore containment is likely to cost much more than $20 million

-Here, you need an argument to establish probability

-Beware of committing this fallacy by going from possibility on top of page 1 to probability on the bottom of page 2

Four Points About Arguments

3. The Straw Man Fallacy

-Misrepresenting the position of the opposing side in such a way that the opposing position appears obviously false or ridiculous.

-Creating a 'straw man' just so you can knock it down.

-E.g., MacLellan hasn’t considered that containment might fail 

Four Points About Arguments

4. The Concorde Fallacy (a.k.a. the Sunk Cost Fallacy)

-What’s wrong with the following argument?

-One reasoning for choosing incineration over containment is that so much money has already been spent on pursuing the incineration option. If we go with containment, that money will all have been wasted.

Four Points About Arguments

The Concorde Fallacy

-Thinking that the fact that a certain amount of money has been spent pursuing option A is by itself a reason for continuing to pursue option A rather than some other option.

-Sunk costs are sunk whether we stick with option A or not.

-Similar arguments were used during the building of the Concorde

Risk

- Risk is inevitable, people won’t always do their jobs well

- Two Phenomena:

-Normalization of Deviance

-A phenomenon to be resisted

-Normal Accidents

-An unavoidable aspect of engineering?

Normalizing Deviance

-Accepting anomalies or increasing boundaries of acceptable risk

-The Challenger case (153-4)

-a long string of acceptance of differences from predicted behavior

-“the joint was officially certified as an acceptable risk, even though the joint’s behavior deviated from design predictions.” (153)

Normal Accidents?

It has been suggested that two features of high-risk systems make accidents both likely and difficult to predict and avoid.

Tight Coupling

-Processes affect one another in a short time

-Little time to correct and contain a failure

Complex Interactions

-Parts of the system can interact in unanticipated ways (150-1)

Are Accidents a ‘Cost of Doing Business’?

According to sociologist Charles Perrow “accidents in complex tightly coupled systems are inevitable.” (151)

Should we simply accept that engineering (and medicine and …) will sometimes lead to catastrophies?

Perceptions of Risk

As MacLellan pointed out, there is a difference between how experts and the public perceive risk

Expert: Risk = hazard+pathway+receptor

Public: Risk = hazard

Likewise, the public and experts often differ when it comes to estimating acceptable degrees of risk

Acceptable Risk

Degree of risk: “the product of the likelihood and the magnitude of the harm.” (155)

-A high likelihood of a minor harm may be a greater risk than a low likelihood of substantial harm

-Rooted in a utilitarian, cost-benefit approach

Acceptable risk: “An acceptable risk is one of where, given the options available, the risk of harm is at least equaled by the probability of producing benefit.” (156)

Limitations on Acceptable Risk

As a utilitarian notion, this idea suffers the usual problems with that view

-Estimating likelihood

-Estimating cost of outcomes

Most would argue it is also in need of deontological limitations

-Suppose project has risk of catastrophic harm to some small group of individuals

-Respect for person may rule out the project even if cost/benefit analysis is OK

Estimating Risk

-In general, people are not good estimators of probability

-In a study asking subjects to estimate risk of harm from smoking, driving, skiing, etc.

-Expert estimates were ~10 times too low

-Public estimate were ~100 times too low (157-8)

Tendency to focus on dramatic cases

-Driving with cell phones vs. fiddling with radio, coffee

Estimating Risk

Risks are seen as greater if caused by humans, than if they have a natural origin. (158)

The Washington sniper?

Risks are discounted over time – an immediate risk is seen as being 30 times greater than a delayed one (158)

Voluntary Risks

“voluntarily assumed risks are seen as inherently less risky, not simply more acceptable.” (158)

“laypeople are generally willing to take risks that are 1000 times … as uncertain as voluntary risks.” (158)

The Public Conception of Risk?

Some argue that the public are not actually bad at estimating risks, they simply do not distinguish between risks and acceptable risks (159)

In other words, the public has a normative conception of risk while experts have a descriptive conception

A Normative Conception of Risk

If we accept this idea, then what is involved in a risk/acceptable risk?

-Consent

-Fairness

-Control

A deontological notion of risk?

Reconsider the Tar Ponds dispute in light of this

The Principle of Acceptable Risk

“People should be protected from the harmful effects of technology, especially when the harms are not consented to or when they are unjustly distributed, except that this protection must sometimes be balanced against (1) the need to preserve great and irreplaceable benefits and (2) the limitations on our ability to obtain informed consent.” (167)

Moral Risk?

To what extent, if any, is a person’s moral status (i.e., how good or bad he is) subject to risk

One view: how good or bad you are is not vulnerable to luck (i.e., risk) at all

How things turn out is irrelevant morally

Is this true?

Legal Luck

The preceding view is not true where the law is concerned. Some actions carry a legal risk

i.e., how much wrong you have done legally is connected to how things turn out

Attempted murder vs. murder

Dangerous driving vs. vehicular homicide

Moral Luck

Some argue that just as we are subject to luck legally, we are also subject to moral luck

-How good or bad we are is dependent on luck

-Contrast utilitarianism vs. deontology on this point

-How troubling an idea is this?