The Rare Disease Test

Why Accurate Tests Still Fool Us

A base rate is how common something is before you run any test. Ignoring it is one of the most expensive reasoning errors there is : and almost everyone makes it.

Here is the scenario. Read it carefully; you will compute the answer on the next screen.

- A disease affects 1 person in 1000 (that is the base rate).

- There is a test that is 99% accurate: if you are sick it says positive 99% of the time, and if you are well it says negative 99% of the time (so it is wrong on 1% of well people).

- You take the test. It comes back positive.

Almost everyone's gut says: 99% accurate, so I am about 99% likely to be sick. Hold that thought. We are going to check it against the base rate instead of trusting the gut.

The method: imagine 1000 people and count.

- Of 1000 people, about 1 is actually sick. The test catches that person : call it 1 true positive.

- The other 999 are well. The test is wrong on 1% of them: 999 x 0.01 is about 10 false positives.

So among everyone who tests positive, you have about 1 true positive and about 10 false positives.

Do the Count

Your Calculation

Use the counts from the tree: about 1 true positive, about 10 false positives.

What the Number Teaches

The Takeaway

The single most important habit from this section: when something is rare, even a very accurate test produces mostly false positives. The base rate can overwhelm the accuracy.

This is not just about medical tests. It is fraud alerts, spam filters, security scanners, and any confident system that flags a rare event. A "positive" from such a system is a reason to look closer, not a verdict. The rarer the thing, the more a single alarm should trigger verification instead of belief.

When Two Things Move Together

Correlation vs Causation

Two things are correlated when they move together : when one goes up, the other tends to as well. It is tempting to conclude that one causes the other. Usually that conclusion is unearned. There are at least three innocent explanations for any correlation:

1. Confounder (a lurking third cause). Ice cream sales and drowning deaths rise together. Ice cream does not cause drowning; hot weather drives both. The confounder is the real engine.

2. Reverse causation. People who use a fitness app are healthier, so the app must cause health? Maybe already-healthy people are the ones who choose the app. The arrow may point the other way.

3. Coincidence. With enough variables measured, some will line up by pure chance with no connection at all.

The professional move: before accepting "X causes Y," ask what else could explain this? A confident source that skips that question is selling you a story, not a finding.

Find the Hidden Cause

Your Turn

A study reports: "Students who own more books score higher on reading tests. Therefore buying more books raises test scores." A shopping app even cites it to sell book bundles to parents, very confidently.

Believing What Fits

The Bias That Feeds Your Beliefs

Confirmation bias is the tendency to notice, believe, and remember evidence that fits what we already think, while ignoring or explaining away evidence that does not. It runs quietly in the background of almost every judgment.

It gets worse with modern tools. Search results, feeds, and recommendation systems learn what you like and hand you more of it. Ask a leading question and you often get a confident answer shaped to agree with you. The result feels like independent confirmation, but it is an echo : the same belief bounced back, dressed up as evidence.

The antidote is deliberate: go looking for the strongest evidence that you are wrong. If you cannot find any, or you dismiss it too fast, that is the bias talking.

Catch the Bias

Your Turn

Maya is sure a certain supplement cured her cold. She remembers the times she took it and recovered, retells those stories, and forgets the times it did nothing. When a friend mentions a study showing it does not work, she says the study "must be biased" and moves on. She also asks a confident health app "is this supplement good for colds?" and it cheerfully agrees.

Independent Verification

The Habit That Ties It All Together

Every tool so far points at one skill: do not trust a confident claim; verify it. The most reliable method is triangulation : check a claim against two or more independent sources before acting on it.

The word independent is doing the heavy lifting. Three sites that all copied the same press release are not three sources : they are one source, echoed three times. Real triangulation needs sources that could disagree but do not.

A practical checklist before acting on any high-stakes claim:

- Independence: Do my sources trace back to the same origin, or were they gathered separately?

- Base rate: How common is this really? Could a confident alarm mostly be a false positive?

- Cause: Is a causal story being sold from what is only a correlation?

- My own bias: Am I believing this partly because I already wanted it to be true?

- Stakes: The bigger the decision (health, money, safety), the more independent confirmation it deserves.

Verify a Real Claim

Your Turn

A study-help app tells you, very confidently and in fluent prose, that a historical date is 1847 and that a formula works a certain way. You have a test tomorrow and the answer matters.

What You Are Taking Away

One Last Thought

You now have four tools against confident error: check the base rate before trusting a rare-event alarm, separate correlation from causation, watch for confirmation bias in yourself and your tools, and triangulate against independent sources before you act.

None of these require you to be a genius. They require one habit: treating confidence : your own or a slick source's : as a question to test, not an answer to accept.

The people who get fooled least are not the ones who feel surest. They are the ones who verify.