Mark Twain popularized the phrase, “There are three kinds of lies: lies, damned lies, and statistics” and it stuck because 125% of humans who have ever lived have been misled using statistics. (See what I did there?) If you think this is a problem unique to politics or baseball, think again:
Reinhart is a physicist turned statistician who has set out to write a book whose aim is to improve the quality of statistical education and understanding that researchers need to have. Statistics Done Wrong is not a textbook. It is a highly informed discussion of the frequent inadequacy of published statistical results and confronts the sacred cow: the p value. Here is what he has to say on page 2.
“Since the 1980s, researchers have described numerous statistical fallacies and misconceptions in the popular peer-reviewed scientific literature and have found that many scientific papers — perhaps more than half — fall prey to these errors. Inadequate statistical power renders many studies incapable of finding what they’re looking for, multiple comparisons and misinterpreted p values cause numerous false positives, flexible data analysis makes it easy to find a correlation where none exists, and inappropriate model choices bias important results. Most errors go undetected by peer reviewers and editors, who often have no specific statistical training, because few journals employ statisticians to review submissions and few papers give sufficient statistical detail to be accurately evaluated.”
Astonishing to my eyes was his conclusion that
“The methodological complexity of modern research means that scientists without extensive statistical training may not be able to understand most published research in their fields.”
The excerpt above is from a post on Boing Boing about a book that Alex Reinhart has written (Statistics Done Wrong) to try and address the issue of statistical malpractice and I’m thinking it could be a useful reference for all of us who need to parse statistics as part of our daily lives, which based on my extensive research would be all 125% of us.