Nutrition's Freakonomics
Nutrition needs it's own version of Steven Levitt.
In case you don't recognize the name, Steve Levitt is the brilliant "rogue" economist who wrote the highly recommended book "Freakonomics". Levitt is known for examining all sorts of data that economists rarely look at and drawing fascinating conclusions about what, for example, street drug dealers really earn, or the relationship between the Klu Klux Klan and real estate agents.
I was thinking about this today when I read some new statistics on Iraq. Or, more correctly, when I read about the impossibility of getting good statistics about Iraq. (Hold on, this really is going to come back to nutrition and health.) Seems we now "know" that there are about 348,000 well-trained Iraq troops ready to provide security. But when you look a little closer at who these troops are, you find that while 348,000 may have signed up, the real number of troops is possibly a couple hundred thousand less. Many of these troops don't show up for duty, many aren't trained, and many have divided loyalties and even fight for the other side.
Which brings me to how important it is to examine where statistics come from, or, even more essentially, how the numbers are defined.
For example, if I tell you 20% of the population is on a low-carb diet- what exactly does that mean? How are we defining a low-carb diet? Without a definition of terms, these- or any other- stats don't tell you very much. (For example, if the "low carb diet" they're following is defined by the American Dietetic Association, it could mean any amount of carbs less than 55% of calories; if it's defined by, say, the Atkins Induction Phase it could mean 20 grams or less a day. Those two definitions aren't even in the same universe, but if you don't do a Steven Levitt on the stats, you could wind up just accepting the 20%-of-the-population figure without having a clue as to what it really means.) If I tell you 27% of the population is concerned about health, what exactly does that mean? How did the pollster decide who goes into that category? Did he use behavioral measures? Or did he just ask "is health important to you"? When you read that a school is "improving" in their test scores what does that mean? That instead of half their students failing, only 40% are failing? Or does it mean that more of their graduates are going to Harvard?
I've been guilty of this kind of creative use of statistics myself. Or, to be fair, this kind of creative use of stats has been used on my behalf, as it probably has on anyone trying who's ever had a book deal or a publicist. When my first book, "Jonny Bowden's Shape Up! The eight week program to transform your body your health and your life" came out, there was much talk about how 75,000 women had been part of this very successful weight loss program on iVillage. Well, what did that exactly mean? In my case, it meant that 75,000 women had signed up for a little e mail newsletter on the Shape Up Prorgram; how many of those actually did the program, went to the 8 weeks of online chat meetings, did the assignments and got the results is anyone's guess. The looser definition- signing up for an e mail newsletter- allowed the publicist to make much more dramatic statements.
Definitions are everything.
Is "salmon" a health food? Depends. Are you defining it as "farm raised" or "wild Alaskan"? Big difference. (see my book, "The 150 Healthiest Foods on Earth" to read the whole scoop on this- but you get the point.) What about milk? (Same deal.)
This stuff gets way murkier when it comes to the advertising agencies and big pharma. "Statin Drug X cuts heart attack rates by 33%" sounds great as a headline. When you read the fine print below the ad, it often turns out to be more like this: those with high heart attack risk factors went from 3 per 100 to 2 per 100 in a sample of men over 50 who had already had a heart attack. (No lives were actually spared in the making of this commercial.)
But none of that messy little detail stuff makes for good headlines.
If you're conspiracy minded- and who of us doesn't like a good book depository theory?- it's easy to frame this in the context of "lying with statistics".
But this isn't outright lying I'm talking about.
It's concealing.
And concealing with statistics will continue as long as we have numbers. It's our job as smart consumers of information to dig a little deeper into the murkier universe of definitions.
Steven Levitt, if you're listening- how about taking on big pharma?
In case you don't recognize the name, Steve Levitt is the brilliant "rogue" economist who wrote the highly recommended book "Freakonomics". Levitt is known for examining all sorts of data that economists rarely look at and drawing fascinating conclusions about what, for example, street drug dealers really earn, or the relationship between the Klu Klux Klan and real estate agents.
I was thinking about this today when I read some new statistics on Iraq. Or, more correctly, when I read about the impossibility of getting good statistics about Iraq. (Hold on, this really is going to come back to nutrition and health.) Seems we now "know" that there are about 348,000 well-trained Iraq troops ready to provide security. But when you look a little closer at who these troops are, you find that while 348,000 may have signed up, the real number of troops is possibly a couple hundred thousand less. Many of these troops don't show up for duty, many aren't trained, and many have divided loyalties and even fight for the other side.
Which brings me to how important it is to examine where statistics come from, or, even more essentially, how the numbers are defined.
For example, if I tell you 20% of the population is on a low-carb diet- what exactly does that mean? How are we defining a low-carb diet? Without a definition of terms, these- or any other- stats don't tell you very much. (For example, if the "low carb diet" they're following is defined by the American Dietetic Association, it could mean any amount of carbs less than 55% of calories; if it's defined by, say, the Atkins Induction Phase it could mean 20 grams or less a day. Those two definitions aren't even in the same universe, but if you don't do a Steven Levitt on the stats, you could wind up just accepting the 20%-of-the-population figure without having a clue as to what it really means.) If I tell you 27% of the population is concerned about health, what exactly does that mean? How did the pollster decide who goes into that category? Did he use behavioral measures? Or did he just ask "is health important to you"? When you read that a school is "improving" in their test scores what does that mean? That instead of half their students failing, only 40% are failing? Or does it mean that more of their graduates are going to Harvard?
I've been guilty of this kind of creative use of statistics myself. Or, to be fair, this kind of creative use of stats has been used on my behalf, as it probably has on anyone trying who's ever had a book deal or a publicist. When my first book, "Jonny Bowden's Shape Up! The eight week program to transform your body your health and your life" came out, there was much talk about how 75,000 women had been part of this very successful weight loss program on iVillage. Well, what did that exactly mean? In my case, it meant that 75,000 women had signed up for a little e mail newsletter on the Shape Up Prorgram; how many of those actually did the program, went to the 8 weeks of online chat meetings, did the assignments and got the results is anyone's guess. The looser definition- signing up for an e mail newsletter- allowed the publicist to make much more dramatic statements.
Definitions are everything.
Is "salmon" a health food? Depends. Are you defining it as "farm raised" or "wild Alaskan"? Big difference. (see my book, "The 150 Healthiest Foods on Earth" to read the whole scoop on this- but you get the point.) What about milk? (Same deal.)
This stuff gets way murkier when it comes to the advertising agencies and big pharma. "Statin Drug X cuts heart attack rates by 33%" sounds great as a headline. When you read the fine print below the ad, it often turns out to be more like this: those with high heart attack risk factors went from 3 per 100 to 2 per 100 in a sample of men over 50 who had already had a heart attack. (No lives were actually spared in the making of this commercial.)
But none of that messy little detail stuff makes for good headlines.
If you're conspiracy minded- and who of us doesn't like a good book depository theory?- it's easy to frame this in the context of "lying with statistics".
But this isn't outright lying I'm talking about.
It's concealing.
And concealing with statistics will continue as long as we have numbers. It's our job as smart consumers of information to dig a little deeper into the murkier universe of definitions.
Steven Levitt, if you're listening- how about taking on big pharma?
0 Comments:
Post a Comment
Links to this post:
Create a Link
<< Home