52: Give p's a chance (with Daniel Lakens)

In this episode, Dan and James welcome back Daniel Lakens (Eindhoven University of Technology) to discuss his new paper on justifying your alpha level. Highlights: Why did Daniel write this paper? Turning away from mindless statistics Incremental vs. seismic change in statistical practice The limitations to justifying your alpha The benefits of registered reports Daniel’s coursera course What’s better? Two pre-registered studies at .05 or one unregistered study at .005? Testing at the start of semester vs. the end of semester Thinking of controlling for Type 1 errors as driving speed limits Error rates mean different things between fields What if we applied the “5 Sigma” threshold used in physics to the biobehavioral sciences? What about abandoning statistical significance How did Daniel co-ordinate a paper with 88 co-authors? Using time zones to your benefit when collaborating How can junior researchers contribute to these types of discussions? Science by discussion, not manifesto The dangers of blanket recommendations How do you actually justify your alpha from scratch? Links Daniel on Twitter - https://www.twitter.com/lakens Daniel’s courser course - https://www.coursera.org/learn/statistical-inferences Justify your alpha paper - https://psyarxiv.com/9s3y6 Abandon statistical significance - https://arxiv.org/abs/1709.07588 Using the costs of error rates to set your alpha - https://doi.org/10.1111/j.1461-0248.2004.00625.x Special Guest: Daniel Lakens.