If you’ve ever tried to have an argument about anything related to science, this has probably been thrown in your face a time or two. But what does it mean anyway?
It means that is hard to prove anything when it comes down to science. Let’s say you switch your dog to a raw food diet, and your dog improves in health after switching him. While it is true they correlate, because one happened right after another, it is not necessarily true that the raw food diet caused the dog to be healthier.
It could be that the kibble you fed before was so crappy anything you switched to would have been better, or that the dog was ill from some other cause and got better on his own. It might even be that raw food is expensive, and so you feed less, thus slimming your dog down and it’s the weight loss that caused the sudden boost in health.
In order to prove whether it was the raw food that made your dog healthier, or some other factor, you need to do a controlled study.
A controlled study involves a lot of components, including a comparison group of dogs that do not change in diet, and ideally all the dogs in both groups would be the same age, health, etc., so that preexisting conditions, normal aging, or breed specific factors are eliminated as much as possible. This is also why it needs to be a relatively large group. The larger the group, the less the overall results will be changed by factors beyond your control.
Scientists, just like regular people, are plagued with Confirmation Bias, and humans that are being studied can fall prey to the placebo effect, making a blind or ideally a double blind study important to control these problems. (In a blind study, the patients don’t know if they are in the control group or not. In a double blind, the scientists don’t know either. For dogs, this isn’t a terribly big issue because they don’t even know they are in a study at all.)
People like you and me, just dabbling in experiments at home, may not be able to account for all of these problems, and we may not be able to check our own work well enough to avoid issues such as Simpson’s Paradox, where looking at small bits of evidence instead of the whole picture can reverse the results of your findings.
I’m not really a scientist. I’m just a housewife that finds the home a fascinating place to explore. I can conduct my tests, and hazard guesses as to what they might mean, but they are not blind studies, my study group is limited to the pets and people in my house, and I do suffer from confirmation bias. (After all, I do believe that the raw food diet mentioned before helped my dog, even though the evidence is purely anecdotal.)
My encouragement to you would be to take my experiments with a grain of salt, to conduct your own tests, and to draw from them your own conclusions. I also suggest that you keep your mind open no matter where that takes you.
If we are to grow as a society we need to focus on being open to the world around us. That is the only way we will be able to achieve greatness.