Taking Jeff Masters to statistics school. While Jeff is a meteorologist, I really am starting to think that many of the “scientists” doing climate research are second raters. If not third.
4 thoughts on “Mastering The Heat-Wave Puzzle”
Comments are closed.
I’m afraid that Willis Eschenbach is no better than Jeff Masters.
The Poisson distribution is my favourite. I pull it out on a regular basis to point out the lack of substance and statistical signifigance of news stories about the number of deaths on the road, or suicides in the army, or the like.
The great thing about the Poisson distribution is that even if all you know is the mean (or even just one sample), you can immediately know that the standard deviation should be the square root of the mean, and therefore quickly calculate the (say) 95% confidence interval of two standard deviations around the mean. This is enough to annihilate most breathless reporting as not having made its case.
Poisson is totally appropriate for statistics that have a mean very small compared to the population, can not be negative, and have a theoretical possibility (however unlikely) that it could happen to anyone or everyone. Everyone could commit suicide this year. They just don’t, most of them.
The problem with Willis Eschenbach’s use of Poisson here is that a Poisson distribution with lambda = 5.213 predicts not only that in 1374 samples you can expect to find 13 sucesses 2.5 times. It also predicts that you’ll have 14 successes 0.94 times. And 15 sucesses 0.3 times. And 16 successes 0.1 times.
In fact you should find a total of 3.94 13 month stretches with more than 13 months in the top third temperatures for that month.
Huh?
Of course this is nonsense. You can’t have 15 out of 13 months with temperatures in the top one third. 13 is a hard maximum.
The indisputable fact that Eschenbach’s model vastly overestimates the probability for N > 13 (they should all be exactly zero) might lead one to suspect that it also overestimates for N = 13. And one would be correct.
It’s pure fluke the Eschenbach’s model looks about right. It’s wrong.
Jeff Masters’ use the the Binomial distribution and answer of 1 chance in 1.6 million is dead right, under the assumption that the probabilities for sequential months are independent.
It’s the assumption of independence that is the problem here.
If you picked 13 months AT RANDOM from the 1374 months on offer then I’m confident that all thirteen would be in the hottest third one time in 1.6 million.
(actually it will be 1 time in about 2 million if you aren’t allowed to pick the same month multiple times. Hypergeometric distribution with population = 1374, population successes = 458 (one third), sample size = 13, sample successes = 13)
(If I can find the actual data I’ll blog on this, and do the statistics properly. A simple yes/no for each month will be enough)
“Jeff Masters’ use the the Binomial distribution and answer of 1 chance in 1.6 million is dead right…”
Actually, it trivially isn’t. As a lower bound, there are about 100 non-overlapping, independent 13 month time windows. As the probability is small, the odds of getting at least one of those to come up in the upper 1/3 is about 100 times Master’s probability (1 – (1 – 1/3^13)^100). So, even with a very conservative lower bound, he’s off by at least two orders of magnitude.
I very much doubt Master’s calculation is right, but I cannot muster up any motivation to check it. I mean, it’s pretty much self evident that, in a warming or warmed planet, you will have increasing numbers of days over a given threshold. And, nobody of any consequence believes that the globe didn’t warm in the latter third of the 20th century (though that pretty much stopped about a decade ago).
The question is not “has the Earth warmed,” but “why did it warm, and is there anything we can do about it?” The answer to the first is “reply hazy, ask again later”, and to the second is “no”.
The “heat wave puzzle” sounds like an obscure unsolved problem by a 19th century physicist.