The Food News Hoax
We strive to provide a fairly comprehensive look at the world of fitness and nutrition, keeping you up-to-date on all the latest and most interesting studies. But, and we have admitted it more than once, we are not scientists. And even we are vulnerable to repeating a study that may have flaws. Lucky for us, we missed the purposefully flawed study that said chocolate makes you thinner. But it does remind usÂ how we need to read everything with a critical, skeptical eye, especially if it’s such good news.
The article, written by the person who created the hoax, is pretty long and entertaining, but here are the choice nuggets:
1. A prankster put together a “legitimate” study by taking a very small sample size of people, measuring various aspects and then feeding some of them chocolate for 21 days. ‘
2. They massaged the data, something called p-hacking, which essentially allows you to massage the data to show whatever you want.
3. They paid to have a “medical journal” publish it who clearly didn’t peer-review it whatsoever.
4. They started up the PR machine.
5. Without asking any critical questions, some news agencies took the bait and wrote articles about how chocolate is great for you and can help you lose weight.
6. Places like the NY Times and NPR didn’t publish it, but others did while doing the absolute minimum of fact-checking, if they checked anything at all. Once they saw the headline the study could generate they published it without checking to see what it actually said.
7. The hoax was revealed but the damage had already been done.
So what did we learn? Well, again the article is very informative, but mostly it shows that whenever you see a new study that shatters old thinking, allows you to do something that was once thought unhealthy or is a guaranteed weight-loss cure, then you should probably be looking at the study yourself or at least waiting to see if some more legitimate outlets support the findings. As the hoaxer says, “For far too long, the people who cover this beat have treated it like gossip, echoing whatever they find in press releases. Hopefully our little experiment will make reporters and readers alike more skeptical.”
And we’ll try and be better — we promise.