Winning an Oscar, living longer, and the strange idea of a p value

“Win an Oscar, live longer” said the Sunday Times headline on 27 February 2011. Oscar winning actors, apparently, live 3.9 years longer than other actors. Presumably Daniel Day-Lewis, with his three Oscars, has booked himself an additional 12 years to savour his success! 

This was based on an article, Survival in Academy Award–Winning Actors and Actresses, published in the Annals of Internal Medicine in 2001. How can we be sure this is right? The statistic given in the article to answer this question is p = 0.003. This is the so-called p value and is the standard way of describing the strength of evidence in statistics. 

The p value tells us that the probability of the observing data as extreme as this (from the perspective of winners surviving longer than non-winners), on the assumption that winning an Oscar actually conferred no survival advantage at all, is 0.003, so there must be something about winning an Oscar that makes people live longer. Obviously the lower this p value is the more conclusive the evidence for winners living longer.

Confused? Is this really obvious? The p value is a measure of the strength of the evidence that does not tell us how likely the hypothesis is to be true, and has the property that low values indicate high levels of certainty. But this is the system that is widely used to report the results of tests of statistical hypotheses. 

Another way of analyzing the result would be to say that the evidence suggest that we can be 99.85% confident that Oscar winners do, on average, live longer – as suggested in “P values, confidence intervals, or confidence levels for hypotheses?”. This seems far more straightforward, but nobody does it this way. P values dominate, despite, or perhaps because of, their obscurity.

There is another big problem with this research. In 2006 the journal published another article, “Do Oscar Winners Live Longer than Less Successful Peers? A Reanalysis of the Evidence”, pointing out a major logical flaw in the research design. Actors who live a long time obviously have more chances to win an Oscar than those who die young. The authors cite an 1843 study pointing out “the greater longevity of persons who reached higher ranks within their professions (bishops vs. curates, judges vs. barristers, and generals vs. lieutenants).” The original study failed to take account of this; when this factor is taken into account, the additional life expectancy is only one year and the confidence that winners will live longer is 93% (which is conventionally not considered statistically significant). This is obviously a separate problem to the p value problem, but it does make me wonder whether obscure statistics, of which the p value is just a minor part, can help researchers hide the logical flaws in their study, perhaps even from themselves.

Even more worryingly, the Sunday Times article claiming Oscar winners live longer was published five years after the article challenging the original research, and included a quote from the author of the original research saying that they get “more invitations to cool parties. Life is better for Oscar winners.” Why let truth get in the way of a good story?

The curse of deep skepticism

My problem is that I don’t believe in most of the things I am supposed to hold dear as an academic. Few of the usual assumptions about what is worth teaching and researching make sense to me. Values like the importance of striving for the truth (what’s that?), and of preserving academic “standards” seem, to me, misguided, meaningless, or at best, over-rated. I can’t get worked up about plagiarism like most of my colleagues – why should using the exact phrase used by someone else be such a sin, and if your first language isn’t English, isn’t this the obvious thing to do? I have never been able to join groups with common, comfortable assumptions about what is worthwhile, and to get on in academia that is what you need to do. I pay lukewarm lip service to some of it – like the conventional methods of statistics as an approach to research, like papers published in top journals being a gold standard for research, like the value of a first class degree, and so on – but I can never actually believe in much of it. 

This blog is an attempt at therapy for my condition. It is intended to articulate some of my skeptical thoughts, to help me understand what I really think. Obviously, I am not expecting many readers, but if you are reading this then I am pleased, and would appreciate your comments, even, perhaps especially, critical ones. (Incidentally, many of the details, like the project on leadership and the third age, are fictional, but the spirit of my comments is deeply felt.)