The other day on the subway I saw an ad that claimed “Only 3% of kids’ meals options are healthy.” Of course this was disturbing, but then I started wondering, What does this claim actually mean, and how could it possibly have been generated?
We see these sorts of claims all the time. Advertisers and influencers know that numbers are powerful and hard for the average person to refute. As a researcher, I’ve learned to view everyday stats with a healthy dose of skepticism, especially when I encounter highly subjective terms like “healthy,” “safe,” “clean,” and “natural.”
The 3% claim raised my suspicion for a number of reasons. What determines a “healthy” meal option? Who decided this? And what are these meal options? The more I thought about it, the more “3%” seemed like a subjective claim, not a rigorously sourced fact.
To test my intuition, I surveyed 250 people nationwide about how healthy their diet is.
Almost 2/3 of respondents reported that their diet is “very healthy” or “somewhat healthy.” But when I asked for their definition of “healthy,” I got wildly divergent responses, touching on everything from amount and types of foods/beverages to a general view on eating. Some said “healthy” meant eating a lot of fruits and vegetables. Others said it meant keeping a low-carb or vegetarian diet. Still others said it had nothing to do with specific foods. “Eating so you feel and look good,” is how one described a healthy diet. Or how about this response: “When you burn more calories than you eat.”
Though “eating vegetables” and “balance” were common themes, I found no agreement on what constitutes a healthy diet. Meaning that the 3% claim rests on a consensus that doesn’t appear to exist. Does it mean kids’ meals have juice, or no juice? A little red meat, or none? Who knows!
Now it may be that kids’ meals deserve a second look, but there’s no way to know from the 3% claim. If only researchers had asked exactly how much meat or juice or carbs were consume—tangible, objective figures—rather than relying on subjective assessments!
George Orwell wrote that meaning should determine which words we use, not the other way around. I feel the same way about statistics. The honest presentation of facts is just as important as the rigor of the surveys that underpin them. Survey data shapes our conception of the world and informs our choices as business owners and consumers. And if we’re going to make good choices, we need to know the facts, fairly presented.