nu — Judgment Under Uncertainty: Expected Value, Calibration & Bias

Welcome

Welcome to Judgment Under Uncertainty.

Every important choice you will ever make is a bet placed with incomplete information. You rarely know how things will turn out. What you can control is the quality of your reasoning before the result arrives.

Today you will build five tools that separate careful decision-makers from lucky ones: expected value, calibration, bias awareness, reversibility, and the difference between a good decision and a good outcome.

These tools stay useful even when an automated system hands you a recommendation, because the responsibility for the call still rests with you.

Weighing Payoffs by Probability

Expected value comparison of two options with variance spread

Expected Value

When an outcome is uncertain, expected value (EV) is the average payoff you would get if you could repeat the same decision many times.

The formula is a weighted sum: EV = sum of (probability x payoff). Multiply each possible payoff by its probability, then add the products together.

Work through two options:

- Option A (safe bet): 90% chance of +$100, and 10% chance of -$50.

EV(A) = 0.90 x (+$100) + 0.10 x (-$50) = 90 - 5 = +$85.

- Option B (long shot): 30% chance of +$400, and 70% chance of -$50.

EV(B) = 0.30 x (+$400) + 0.70 x (-$50) = 120 - 35 = +$85.

Both options have the same expected value: +$85. On raw EV alone, they tie. In a moment you will see why a careful decider might still prefer one over the other.

Compute an Expected Value

Your Turn: One Number

Here is a single bet. Two outcomes, and the payoffs are already stated as net dollars:

- 10% chance of a net +$75

- 90% chance of a net -$5

What is the expected value of this bet, in dollars? Show the products if you can, then give the final number.

When Risk Beats Raw EV

Expected value assumes you can repeat the bet many times so the average wins out. Many real decisions happen once. A single loss you cannot recover from can end the game before the average ever shows up.

Variance measures how spread out the possible payoffs are. Option A earlier stayed in a narrow band; Option B could swing far in either direction. Same EV, very different risk.

Rule of thumb: when a possible loss is large enough to ruin you (bankruptcy, injury, losing something irreplaceable), let that risk override raw EV. A slightly lower average is a fair price for avoiding ruin. This is why people buy insurance even though insurance carries a negative EV.

Compare Two Options

Your Turn: Two Options

A small company must choose one launch plan:

- Plan A: 60% chance of +$50,000 profit, and 40% chance of a -$20,000 loss.

- Plan B: a guaranteed +$15,000 profit.

Compute the expected value of each plan, state which plan has the higher EV, and explain when the company should still prefer the other plan.

What Confidence Should Mean

Calibration: What Confidence Should Mean

When you say you are 70% confident, you are making a testable claim: across all the times you feel exactly this sure, you should turn out right about 70% of the time.

A well-calibrated person who says 70% on 100 predictions gets about 70 of them right. If they get only 40 right, they were overconfident. If they get 90 right, they were underconfident.

Overconfidence is the most common calibration error. Most people are far more certain than their track record justifies, which quietly wrecks their decisions. The fix is to track your predictions and compare your stated confidence to your actual hit rate.

Reading Your Own Confidence

Your Turn

Suppose you make 10 separate predictions, each time saying you are 70% confident.

If you are well calibrated, about how many of the 10 should come true? And what does it mean about your confidence if only 3 of the 10 turn out correct, versus if 9 of the 10 turn out correct?

Three Biases That Hijack Decisions

Even with EV and calibration in hand, predictable mental shortcuts distort judgment. Three of the most damaging:

1. Sunk-cost fallacy: letting money, time, or effort you already spent (and cannot get back) keep you committed. "We have already invested two years, we cannot stop now." That past spending is gone either way; only future costs and future benefits should drive the choice.

2. Overconfidence: overrating how much you know and how likely you are to be right (the calibration error from the last section). It makes you skip backups, underestimate timelines, and ignore warning signs.

3. Anchoring: locking onto the first number you hear and adjusting too little from it. The first price named in a negotiation, or the first estimate on a project, drags every later judgment toward itself even when that first number was arbitrary.

You cannot delete these biases, but naming them lets you catch them in the act.

Spot the Sunk Cost

Your Turn

Consider this situation:

You have already spent $60 and three hours on a project that is clearly going nowhere. A teammate says: "We have put so much into this, we cannot quit now."

Which bias is your teammate falling into? Name it, and explain what should actually drive the decision about whether to continue.

One-Way and Two-Way Doors

Not every decision deserves the same care. A useful test: is this a one-way door or a two-way door?

Two-way doors are reversible. If the choice turns out wrong, you walk back through and undo it cheaply: trying a new tool, rearranging a room, testing a study schedule. Decide these fast. Over-deliberating a reversible choice wastes time you could spend learning from the result.

One-way doors are irreversible, or expensive to reverse: deleting the only copy of a file, dropping out of school, a permanent tattoo, publishing something to the whole world. Decide these slowly and carefully, gather more information, and build in safeguards.

The skill is matching your deliberation to the reversibility. Slow decisions on two-way doors and fast decisions on one-way doors are both expensive mistakes.

Sort the Doors

Your Turn

Here are two decisions:

- (a) Permanently deleting the only copy of a file.

- (b) Trying a new seating arrangement in a classroom.

For each decision, say whether it is a one-way (irreversible) door or a two-way (reversible) door, and how much deliberation each one deserves.

Decision Quality vs Outcome Quality

A good decision uses the best reasoning available about probabilities and stakes at the time you decide. A good outcome is simply how things turned out, and luck gets a vote in that.

Because uncertainty is real, a good decision can still lead to a bad outcome, and a reckless decision can get lucky. Judging a decision only by its result is a trap called resulting.

To evaluate a decision fairly, ask: given what was knowable at the time, was this a sensible bet? A poker player can play a hand perfectly and still lose to a lucky card. That does not make the play wrong.

Good Decision or Good Outcome?

Your Turn

A friend ran a red light at 2 a.m. with no cars anywhere in sight, and got home safely.

Was running the red light a good decision? Explain the difference between a good decision and a good outcome, and use it to judge this case.

When a Machine Recommends the Choice

More and more, automated systems will hand you a recommendation: this route, this diagnosis, this trade, this candidate. Often the system is ranking options by expected value or a similar score.

That recommendation is an input, not a verdict. The system optimizes the numbers it was given. It does not feel the weight of a one-way door, it does not know which losses would ruin you, and its probabilities can be wrong, biased, or missing context.

Your job is to weigh what the machine cannot: Is the higher-EV option too risky if it fails? Is this reversible? Do I trust the inputs? Whatever the tool suggests, you weigh EV, risk, and reversibility, and you own the outcome.

Overriding the Recommendation

Your Turn

An automated system recommends Option A because it has the highest expected value.

Under what circumstances should the human override that recommendation and choose differently? Give at least one concrete reason, and say who owns the final decision.