This is a bit off the path of information security but I wanted to share an excellent article on why you should distrust 90% of what you read (including, unfortunately, much of the computer security advice out there).

The Atlantic published this interview with Dr John Ioannidis, a medical researcher who has dedicated his career to showing that “much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong.” This is true even in the ‘gold-standard’ peer-reviewed studies. The biases of funding and publication pressure are too much to overcome. Worse, even when the studies have been overturned, the medical community continues to rely on the old, disproven theories.

While his study and his research are based on medical journals and medical research, his findings are applicable to everything from physics to economics to computer science.

You can also read Dr Ioannidis’ original paper at PLoS Medicine. He lays out a detailed mathematical proof that, “assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time.” He wrote a follow-up article here specifically discussing the distortion caused by publication practices. I recommend both for anyone with an interest in the scientific method and/or an interest in sorting truth from rumor among the deluge of “good advice” on the internet.

Leave a Reply