Wine drinkers and chocolate lovers love it when a new study comes out claiming that wine or chocolate is good for us, “yippee! I don’t have to feel guilty about eating something I love.”
This is a natural reaction, nobody likes to feel guilty and we want to be able to justify our actions. However it seems like we are getting a little too good at extrapolating, being biased and failing to see the big picture. For instance, the cocoa bean has health benefits but how many of you chocolate lovers are eating 100% cocoa powder? Hot chocolate, milk chocolate, chocolate candies and chocolate chip cookies are not the chocolate with health benefits. The scientific community is still debating whether wine is healthy. Good science seems to tell us that abstaining is healthier than consuming one glass of wine a day but there is good science telling us that drinking one glass a day will not harm us and may provide some health benefits. Can drinking wine reduce our risk of heart disease? When BBC’s Trust Me I am a Dr team asked “How can I avoid getting heart disease?” they found that “The evidence on alcohol at the moment suggests that it is not protective against heart disease.”
Click here to read a great article discussing both the pros and cons of alcohol consumption.
We are too quick to leap onto a study that tells us what we want to hear and it is our tendency to make that leap that I want to call to your attention. Being aware of your bias is critical in being able to evaluate good science. I love sugar and I do not like alcohol. I want to believe that alcohol is bad for us. I think wine is a horrible thing to do to a grape and chocolate cake is a beautiful thing to do to cocoa. I am fully aware of my bias so when I am reading studies, instead of looking at what the study says I first ask, IS IT A GOOD STUDY?
THE RULES OF GOOD SCIENCE
When it comes to distinguishing which studies are good or which studies are questionable look for the following;
Randomized-A Randomized Control Trial (RCT) is a study in which the participants are assigned by chance to separate groups; neither the researchers nor the participants can choose which group. (PubMed Health Glossary-Source: NIH – National Cancer Institute).
Peer Reviewed– A Scholarly peer review (also known as refereeing) is the process of subjecting an author’s scholarly work, research, or ideas to the scrutiny of others who are experts in the same field, before a paper describing this work is published in a journal or as a book. (Wikipedia).
Double Blind-A double-blind study is one in which neither the participants nor the experimenters know who is receiving a particular treatment. This procedure is utilized to prevent bias in research results. Double-blind studies are particularly useful for preventing bias due to demand characteristics or the placebo effect. (Verywell.com).
It would be a good idea to become familiar with these terms as well,
Anecdotal Evidence-Evidence from anecdotes, i.e., evidence collected in a casual or informal manner and relying heavily or entirely on personal testimony. (Wikipedia).
Dictionary.com defines anecdotal evidence as non-scientific observations or studies, which do not provide proof but may assist in research efforts.
Cherry Picking-Cherry picking involves suppressing evidence because a company or person selects which studies it is going to reference to support their claim, or to confirm a particular position while ignoring a significant portion of related cases or data that may contradict that position. (Wikipedia).
Extrapolate-to infer or extend the application of (a method or conclusion) to an unknown situation by assuming that existing trends will continue or similar methods will be applicable. (Oxforddictionaries.com).
In addition look for studies that are current and that have been reproduced several times, by different researchers.
This means you have to spend a little bit of time verifying what you read. Before I read an article I do a quick search on the author; what is their background and who is paying them? A friend of mine follows the Paleo diet and I am skeptical of this diet as it eliminates legumes, a very healthy food group and a staple in my diet so I did a quick Google search “paleo diet randomized studies.” I found some interesting articles, this one was particularly good, it examined several studies on the paleo diet. Adding these words to your vocabulary gives you a new language allowing you to be more critical and informed. Once you know the words to use in your searches it becomes a lot easier to weed out the “bad science” and become aware of the “media’s habit of oversimplifying and overstating study results” (American Journal of Clinical Nutrition, “Paleo diet still lacks evidence.”)
Let’s take a closer look at a couple of diets and examine the science behind their claims.
I have read Wheat Belly and it has affected how I eat. It motivated me to reduce my overall intake of white flour and grains. Vegetables are my staple, grains are a treat. However, Wheat Belly breaks several of the rules of good science. This book is full of anecdotal evidence, Dr. Davis claims that the wheat of today is a “frankenwheat”and that our modern wheat is not only unhealthy and toxic, but also addictive. The “Frankenwheat” notion has never been scientifically proven. Dr. Davis uses a study on rats from 1979 to support his claim of wheat being addictive and that study is not recent and has not been peer reviewed or replicated. This CBC article summarizes the shaky science behind Wheat Belly.
In a recent BBC Horizon Episode: Clean Eating Dr. Giles Yeo spoke with Dr. Davis who is now extrapolating from Dr. Fasano‘s research on Celiac disease to support his claim that, “the consumption of grains is harming all of us to some degree.” A claim which has not been scientifically proven.
- Dr. Davis uses the increase in Celiac Disease to support his position, however the increase in cases of this disease is mostly attributed to, according to Dr. Fasano, our improved ability to detect and diagnose the disease.
- In addition Dr. Fasano’s research shows that you need to have some pre-existing conditions (four in fact; genetic predisposition, a leaky gut, a faulty immune system AND an imbalance in gut microbes) in order for Gluten to be a problem.
- Finally, Dr. Fasano disagrees with Dr. Davis’ claim that the Gliadin protein can initiate the process of an autoimmune response leading the body to attack its own organs. Dr. Davis is not only extrapolating to support his philosophy but ignoring good science (cherry picking).
Dr. Bill Davis is particularly dangerous because he portrays himself as someone, like me, who is trying to wade through all the misinformation out there and give people the facts. He is guilty of questionable science. Dr. Giles Yeo summarizes Dr. Davis and his philosophy like this “Bill Davis has no proof that giving up grains will have all these health effects on this wide range of diseases. Extreme dietary advice requires proof otherwise all you are doing is stoking fear about a food group that most people shouldn’t have to worry about.” BBC Horizon: Clean Eating.
The China Study
The China Study is known as the “most comprehensive study of nutrition ever conducted” however it too has been found to have used questionable research methods. The main author, Dr. Colin Campbell asserts that “plant based foods are beneficial and animal based foods are NOT, period.” However eliminating ALL animal based foods has not been scientifically proven to be a healthier way of eating.
Dr Giles Yeo asked if Dr. Colin could have been affected by Observational Bias- when a researcher sees what they want to see instead of what the data shows, while researching for the China Study. Dr. Colin does admit that this may have occurred. In addition Dr. Colin used Cholesterol as a Proxy;
- Proxy Method-in which a variable (Dr. Colin used increased levels of Cholesterol) which correlates with the variable of interest (Increased risk of cardiovascular disease) is measured, and then used to infer the value of the variable of interest. (Wikipedia).
- Proxy Method, is a form of Extrapolation-A= Meat Intake, B= Increased cholesterol levels and C= Increased risk of cardiovascular disease. If you have data linking A to B and B to C, you cannot then conclude that A=C. Many factors contribute to increased cholesterol, like genetics, not just meat intake.
Dr. Colins went into his research looking to prove that meat consumption increased the risk of CV disease, he picked the data on cardiovascular disease that supported his position instead of considering all of the risk factors for cardiovascular disease.
A new language
I want to give you a new language which allows you to read articles, journals and studies with a critical mindset. I found some useful tips on how to tell fake news from real news and the tips apply here, especially this one, How does it make you feel? Is the study/article telling you what you WANT to hear? Is it confirming your bias? It is important to be aware of your biases otherwise it is too easy to fall prey to extreme health claims that ultimately lead to disappointment, frustration and not knowing who to trust and what to believe. Reading a study published in a scientific journal can be intimidating but you can save yourself time by looking for these key words; randomized, peer reviewed, and double blind. Knowledge is your best weapon against media hype and bad science.