Winning an Oscar, living longer, and the strange idea of a p value

“Win an Oscar, live longer” said the Sunday Times headline on 27 February 2011. Oscar winning actors, apparently, live 3.9 years longer than other actors. Presumably Daniel Day-Lewis, with his three Oscars, has booked himself an additional 12 years to savour his success! 

This was based on an article, Survival in Academy Award–Winning Actors and Actresses, published in the Annals of Internal Medicine in 2001. How can we be sure this is right? The statistic given in the article to answer this question is p = 0.003. This is the so-called p value and is the standard way of describing the strength of evidence in statistics. 

The p value tells us that the probability of the observing data as extreme as this (from the perspective of winners surviving longer than non-winners), on the assumption that winning an Oscar actually conferred no survival advantage at all, is 0.003, so there must be something about winning an Oscar that makes people live longer. Obviously the lower this p value is the more conclusive the evidence for winners living longer.

Confused? Is this really obvious? The p value is a measure of the strength of the evidence that does not tell us how likely the hypothesis is to be true, and has the property that low values indicate high levels of certainty. But this is the system that is widely used to report the results of tests of statistical hypotheses. 

Another way of analyzing the result would be to say that the evidence suggest that we can be 99.85% confident that Oscar winners do, on average, live longer – as suggested in “P values, confidence intervals, or confidence levels for hypotheses?”. This seems far more straightforward, but nobody does it this way. P values dominate, despite, or perhaps because of, their obscurity.

There is another big problem with this research. In 2006 the journal published another article, “Do Oscar Winners Live Longer than Less Successful Peers? A Reanalysis of the Evidence”, pointing out a major logical flaw in the research design. Actors who live a long time obviously have more chances to win an Oscar than those who die young. The authors cite an 1843 study pointing out “the greater longevity of persons who reached higher ranks within their professions (bishops vs. curates, judges vs. barristers, and generals vs. lieutenants).” The original study failed to take account of this; when this factor is taken into account, the additional life expectancy is only one year and the confidence that winners will live longer is 93% (which is conventionally not considered statistically significant). This is obviously a separate problem to the p value problem, but it does make me wonder whether obscure statistics, of which the p value is just a minor part, can help researchers hide the logical flaws in their study, perhaps even from themselves.

Even more worryingly, the Sunday Times article claiming Oscar winners live longer was published five years after the article challenging the original research, and included a quote from the author of the original research saying that they get “more invitations to cool parties. Life is better for Oscar winners.” Why let truth get in the way of a good story?