35th Fisher Memorial Lecture: And thereby hangs a tail: the strange history of P-values

Presented by: Professor Stephen Senn - Head of the Competence Center for Methodology and Statistics, Luxembourg Institute of Health
RA Fisher is usually given the honour and now (increasingly) the shame of having invented P-values. A common theme of criticism is that P-values give significance too easily and that their cult is responsible for a replication crisis enveloping science.

However, tail area probabilities were calculated long before Fisher and naively and commonly given an inverse interpretation. It was Fisher who pointed out that this interpretation was unsafe and who stressed an alternative and more conservative one, although even this can be found in earlier writings such as those of Karl Pearson.

I shall consider the history of P-values and inverse probability from Bayes to Fisher, passing by Laplace, Pearson, Student, Broad and Jeffreys, and show that the problem is not so much an incompatibility of frequentist and Bayesian inference, as an incompatibility of two approaches to dealing with null hypotheses. Both these approaches are encountered in Bayesian analyses, with the older of the two much more common. They lead to very different inferences in the Bayesian framework but much lesser differences in the frequentist one. I conclude that the real problem is that there is an unresolved difference of practice in Bayesian approaches. This does not, of itself, constitute a defence of P-values but it does suggest that some of the problems for which they are blamed will not be resolved merely by abandoning them.