If you have watched TV in the past few months you have probably seen the above commercial for 5-hour Energy that would provide a good introduction to misleading media deceptions in a research methods or statistics course. The commercial touts the fact that they surveyed over 3,000 doctors and what they found was amazing. What was apparently so amazing was the fact that 73% of doctors surveyed said that they would recommend 5-hour Energy. Wait, no, that’s not it. 73% of doctors surveyed said that they would “recommend a low-calorie energy supplement to their healthy patients who use energy supplements.” That’s not quite the same thing, is it?
Unlike those “look how great we are” commercials that say things like “four out of five dentists recommend Trident,” the claims made by 5-hour Energy seem more along the lines of “we spent a lot of money to do this survey and we’re going to advertise the results no matter what they show!” In fact, the small print (visible if you enlarge the ad above) is incredibly honest (for a commercial, at least) about the actual methods and findings. Here is the small print in order:
- All doctors surveyed identified themselves as primary care physicians
- Two surveys were conducted to determine the opinions of primary care physicians regarding energy supplements and 5-hour Energy: 1) an online survey of 503 participants; and 2) an in-person survey by 5-hour Energy representatives of 2,500 participants (50% of those approached). In both, participants agreed to review materials regarding 5-hour Energy consisting of label and basic description of its ingredients. Of the 503 online and 2,500 in-person, over 73% said they would recommend a low calorie energy supplement to their healthy patients who use energy supplements.
- Of the 73% of primary care physicians who would recommend a low calorie energy supplement to their healthy patients who use energy supplements, 56% would specifically recommend 5-hour Energy for their healthy patients who use energy supplements.
- Of all primary care physicians surveyed, 47% would specifically recommend 5-hour Energy for their healthy patients who use energy supplements.
So, 5-hour Energy has spent a lot of money on a survey and advertisements to tell people that 27% of doctors would not recommend low-calorie energy supplements to their healthy patients, even if they already use energy supplements. Furthermore, only 47% of the doctors surveyed would actually recommend 5-hour Energy. This is a far cry from the “four out of five dentists” claims. These results are amazing, all right. Amazingly unimpressive!
Memoirs of a SLACer 