"Local farmer experiences a 50% reduction
in his cattle herd."
in his cattle herd."
1. Know your Baseline
With careful questioning, the statistics can then tell a story that is no longer a lie or a damn lie but something that is closer to the truth.
- What if I told you this farmer only had 2 cows in his herd? Doesn't sound so horrific anymore does it? Without knowing where you are starting from it is hard to make a judgment about where you are now.
- A 50% reduction seems significant. It sounds like a really big number however without context it is hard to determine whether it is profound beyond the numbers. A reduction from 2 cows to one cow hardly seems noteworthy. Which leads us to ...
- This is the question I ask to figure out whether we are even measuring the right thing. I'm sure we all think that cows are lovely -- a summer BBQ just wouldn't be the same without them -- but I think we are more concerned with the farmer's livelihood. The reason that statement sounds horrific is because we associated it with the farmer losing half of his income or assets. That may or may not be true but we need to look at....
- What if this farmer was in a Third world country and his entire wealth was two cows? What if the cow that died was the sole source of protein (via milk) for his children? or What if this was a hobby farmer who had cows just to keep the grass down on his half million dollar acreage? Conclusions that we arrive upon have to take into account who the statement originally referred to. We can't always extrapolate from one population to another.
- What were the test scores to begin with?
- Is that an increase that sounds significant only in numbers or in real life too?
- Is test scores what we should be measuring? Should we be measuring something else like success after School?
- Who did they measure this increase in? Is my child in that same environment? Are those results transferable?
With careful questioning, the statistics can then tell a story that is no longer a lie or a damn lie but something that is closer to the truth.