If we want to derive meaning from statistics to gain an accurate world view, we must scan them for bias. People may try to deceive us with how they show statistics, or sometimes the statistics themselves can be deceptive.
One example of this is Simpson's Paradox (also known as Simpson's reversal, Yule–Simpson effect, amalgamation paradox, or reversal paradox.) It is a phenomenon in probability and statistics, in which a trend appears in several different groups of data but disappears or reverses when these groups are combined.
This result is particularly problematic when frequency data is unduly given causal interpretations. The paradox can be resolved when causal relations are appropriately addressed in the statistical modeling.
Let's look at an example. Below is the actual data on survival rates when the Titanic sank.
Based on the data above, can we conclude that a woman is safer in Third Class than in Crew?
Think about this before reading on.
The answer is, no, we cannot. The data is not specific enough. Gender was a key factor for survival rates on the Titanic, so that information needs to be included in the table for us to draw a conclusion about women. The data is inconclusive because gender is a lurking variable.
A lurking variable is a variable that is not included as an explanatory or response variable in the analysis, but it can affect the interpretation of the relationships between variables.
When we see a more complete representation of the data, it is clear that the average woman was far more likely to survive than the average man and that both men and women who were part of the crew were more likely to survive than their counterparts in third class.
We should be very slow to jump to conclusions on the basis of statistical data, especially when the data being measured is not absolutely identical to the question we are asking.