Lies, damn lies, and statistics

At 11:30 this morning, a red headline flashed up on my Bloomberg terminal, trumpeting the breaking news that "AUSTRALIAN EMPLOYMENT FELL 9,700 IN AUG.". The market and currency immediately responded to this negative news by promptly dropping by more than 0.5%.

Barely a day goes without an economic "number" of some sort being released, and each one is closely watched with morbid anticipation by the markets. In this sea of instant information, however, no one seems to pay attention to the fine print. One of the first things you learn in the study of any science is the importance of attaching meaning to all measurements via statistical error bounds. Although often seen as dry and tedious, this process is absolutely vital to establishing the meaningfulness of experimental results.

So, with my developing sense of skeptical for just about everything to do with economics, I thought I'd take a closer look at these numbers. On the website of the Australian Bureau of Statistics you can access their complete press release. Scrolling down a bit, you find a table containing the numbers and their corresponding "95% confidence interval" (this is a bound within which, statistically speaking, the "true" number has a 95% chance of lying; if this bound doesn't include zero, then the measured number can be regarded as "statistically significant" -- i.e. meaningfully different from zero).

Here are the numbers and C.I.s:

Total Employment change: -9 700 (-64 300 to 44 900)

Total Unemployment: 18 400 (-12 600 to 49 400)

Unemployment rate: 0.1% (-0.1% to 0.3%)

Participation rate: 0.0% (-0.4% to 0.4%)


It can be seen that not a one of these is statistically different from zero. Based on these numbers, all we can say with any conviction is that total employment changed by somewhere between -64k and +45k. It is simply meaningless to say that it fell -- the numbers just don't allow us to draw this conclusion.

To make this point perhaps even more clear: if the ABS were to re-run their survey on a different random sample, the above results suggest that there is a decent chance they would observe a reduction, rather than increase, in unemployment.

So why do the markets jerk around like puppets on a string in response to this non-information? My guess is that it's a classic case of people trying to second-guess how other (possibly stupider) people are going to react. In this game, the quality of the underlying information is of little relevance: all that counts is that some people out there will believe it and trade accordingly.

Who's to blame for this madness? The ABS, for not making the limitations of their numbers sufficiently clear (perhaps, but in their defence at least they make the information on confidence intervals freely available on their website if you choose to look)? The media, for seizing on any soundbite-sized piece of information and over-analysing it to death? Or us, for continuing to pay attention? I suspect it's a good dose of each.