An Average Is Not the Whole Story

The Average Hides the Spread

An average (the mean) adds up all the values and divides by how many there are. It gives one tidy number. That tidiness is the danger: one number cannot show how spread out the data is.

The range is the distance from the smallest value to the largest. The spread is how far apart the values sit in general.

Two swimming pools, average depth 1 meter each.

- Pool A: every part is 1 meter deep. Safe to wade.

- Pool B: the shallow end is 0.1 meter and the deep end is 3 meters. The average is still about 1 meter, but a small child could drown in the deep end.

Same average. Completely different reality. The average hid the spread. Whenever you see an average, ask: what is the range, and how spread out are the values?

The Pool Problem

Your Turn

A report says: The average depth of this pool is 1 meter, so it is safe for toddlers.

Three Ways a Graph Lies

A Chart Can Be True and Still Lie

A graph can show correct numbers and still fool you. Three of the most common tricks:

1. Truncated (zoomed) y-axis. An honest bar chart starts its y-axis at zero. A misleading one starts partway up, say at 100. Look at the picture: Brand A sold 102 and Brand B sold 105, a difference of just 3 units. When the axis starts at 100, Brand B's bar looks about 2.5 times taller. Start the axis at 0 and the bars are nearly equal. The data never changed. Only the baseline did.

2. Cherry-picked time range. Show only the slice of dates that supports your story. A stock that fell all year looks great if you only chart the one good month. Ask: why does the chart start and end on those exact dates?

3. Changed or inconsistent scale. Uneven spacing between numbers, or a scale that switches partway, can stretch or squash changes. A line can be made to look like a cliff or a gentle hill using the same numbers.

The detective move is always the same: read the axis and the range before you trust the shape.

Why Is This Graph Misleading?

Spot the Trick

A news chart shows two bars: last month's crime count was 500, this month's is 515. But the y-axis starts at 495 instead of 0. On screen, this month's bar looks roughly four times taller than last month's. The headline reads: Crime Explodes!

Asking a Few vs Asking Everyone

The Sample: A Few Standing In for the Whole

Often you cannot ask everyone, so you ask a sample: a smaller group meant to stand in for the whole group (the population).

A sample only tells the truth about the whole if it represents the whole. When it does not, it is a biased sample.

Examples of biased samples:

- A website asks its own visitors whether they like the website. The people who hate it already left.

- A survey about school lunch is handed out only to students leaving the pizza line.

- A poll runs only during a weekday afternoon, missing everyone at work.

Two questions expose most bias: Who got left out? and Would the left-out people answer differently? A big sample does not fix bias; a poll of a million wrong people is still wrong.

The Survey Problem

Your Turn

A gym posts: We surveyed 200 people at our gym on a Saturday morning. 95 percent say exercise is easy to fit into a busy week. So exercise is easy for almost everyone.

Same Units, Same Baseline

What Makes a Comparison Fair

The last detective move is the simplest: when two things get compared, check that the comparison is fair.

A fair comparison uses the same units and the same baseline for both sides.

Unfair comparisons hide in plain sight:

- Different units. Our phone has 5000 of battery, theirs has only 12! One number is milliamp-hours, the other is hours. They cannot be compared.

- Different baseline. City A had 50 crimes, City B had 200, so City B is far more dangerous. But City B has 20 times the people. Per person, City A is worse. Compare rates (per 1000 people), not raw totals.

- Different time spans. A whole year of one thing against one month of another is not a fair race.

Before you accept any comparison, ask: same units? same baseline? same time span?

The City Problem

Your Turn

A post claims: City A reported 50 thefts last year. City B reported 200 thefts last year. City B is four times as dangerous. City A has 10,000 people. City B has 500,000 people.

What Will You Remember?

One Last Thought

Numbers and charts are not lies by nature. They are tools, and like any tool they can be aimed well or badly. A dashboard, a news graphic, and an automated report can each be technically true and still leave you with a false picture.

You now have four detective moves: read the spread behind an average, check where a graph's axis starts and which range it shows, question who is in the sample, and demand the same units and baseline in any comparison.

Read the axis and the sample, not just the shape.