For some time scientists have been debating how to deal with one of the best-known tools for describing scientific certainty, the notion of "statistical significance." Some think it's good as it is. Others want to make it smaller, while others are in favor of giving it up altogether.
Assuming that we bite the bullet and cut off the concept of probability, we would have to replace it with an even better idea. This Current Issue of The American Statistician Contains Several Opinions – Actual 43:
To paraphrase a famous Winston Churchill quote, the p-value has long been the worst distinction in science … useful scientific ideas on all the other methods that were occasionally tried.
It's not really ps's fault. The value alone simply tells you how likely it is that you supported the wrong horse in your experiment.
If the value falls below 0.05, it usually means that the null hypothesis is less than five percent ̵
Why five percent? Because the story really is like this. It's a better game than 10 percent, but not as strict as one percent. Otherwise, there is really no magic quality.
There are a number of statistical tools that researchers can use to calculate this significance number. Problems arise when we try to translate that mathematical ideal into something that the meat computers in our skulls can actually work on.
Our brain does not handle the probability very well. Maybe it has to do with the fact that we've never looked at the likelihood of being eaten by a bear while he's already chewing our faces.
We treat the sharp separation of a true or false statement significantly better. Thus, a hazy, perhaps <0.05 value is often difficult to swallow, so it can be abused.
"The world is much more insecure than that," said Nicole Lazar, statistician of the University of Georgia, NPR author Richard Harris.
Together With senior director of the American Statistical Association, Ronald L. Wasserstein, and retired vice president of Mathematica Policy Research, Allen Schirm, Lazar wrote an editorial front and anthology of reflections on how we could be better than "p ".
There is a possibility fig can be of use to us, but only if we do not do stupid things with it, assuming that a wise explanation is simply still a competitor to you.
"Knowing what to do with p-scores is necessary, but it's not enough," writes the trio.
"It is as if the statisticians were asking the users of the statistics to rip out the beams and struts that sustain the building of modern scientific research without offering solid building materials."
The articles of the issue are not completely consensus on what these building materials should look like. However, many have some basic elements in common.
The withdrawal of meaning should ideally be translated into tabular data and method descriptions that provide additional nuances and humbly exploit the possibilities while still arguing for a single explanation.
"We must learn to accept uncertainty," write several authors in their Nature statement.
"A practical way is to rename confidence intervals into" compatibility intervals "and interpret them as a way to avoid over-confidence."
This is not just a cheap p-value. The researchers would have to actively describe the practical implications of values within these intervals.
The ultimate goal is to create practices that avoid the clipping that leads to true or false thinking, and instead reinforce the uncertainty underlying the scientific method.
Science is a conversation in their hearts. Politicians, technicians, and engineers are eavesdropping, making the buzz of words a concrete choice, but for scientists taking the next step in research, a p-value alone is not particularly useful.
Unfortunately, it has become a destination in the race for knowledge, with winners awaiting investigations and public awards.
The tipping over of such a firmly established cultural practice becomes much more than a few editorials, and a handful of these require well-argued scholarly work. The p-value has been a respected part of science for about a century, so it will take a while.
But perhaps this way of thinking offers us some practical steps to move beyond statistical significance to a place where blurred lines of uncertainty can be celebrated.