H T M L   T R A N S C R I P T   O F

Non-Trivial Pursuits:
Playing the Research Game

by   B O N N I E   L I E B M A N
 
Director of Nutrition, Center for Science in the Public Interest

facsimile of October 1994 healthletter cover

^ Page 1 of Nutrition Action Healthletter cover story, “Non-Trivial Pursuits: Playing the Research Game” (vol. 21, no. 8, October 1994).

The text of the letter, from reader Claire Howard, to the editor-in-chief of the Nutrition Action Healthletter, Stephen Schmidt, is given below.

[ Text of photographed letter on cover page ]


Dear Mr. Schmidt:

     I give up. I don’t know what to believe anymore with regard to vitamin and mineral supplements.
     For years we have been told that beta-carotene supplements are perfectly safe, and that vitamin E confers many benefits, including control of cholesterol formation, prevention of fibrocystic breast disease, and prevention of hot flashes in postmenopausal women. Now we are cautioned to “shelve the beta-carotene” and reconsider taking vitamin E (June 1994, p. 4).
     How are we health-conscious consumers supposed to react when one well-publicized study contradicts another, earlier, equally well-publicized one? I’m beginning to feel like a ping-pong ball.

Sincerely Yours,

Claire Howard
Bayside, New York
 

[ CSPI’s response ]

Opening quotation markWelcome to the world of science, Ms. Howard. We’re not surprised that you’re confused. It’s not a comforting, here’s-how-the-world-works place, like the textbooks portrayed it in junior high.

“ But you’ll feel less like a ping-pong ball once you see how many — and what kind of — studies researchers have to complete before they can state with reasonable certainty that a food or supplement causes or prevents a disease.

“ Most of those ‘well-publicized studies’ you hear about are just one step in a game we call ‘Non-Trivial Pursuits.’

“ Playing the Research Game

“ ‘New Study Finds Vitamins Are Not Cancer Preventers,’ ran the headline in the New York Times last July.

“ It was the second time in three months that the ‘antioxidant’ vitamins had failed to prevent cancer.

“ In April, a study of 29,000 Finnish smokers found an increased incidence of lung cancer in men taking beta-carotene (vitamin E had no clear effect). And in July, beta­carotene, E, and C failed to prevent precancerous colon polyps in 864 Americans who had already had a polyp removed. [New England Journal of Medicine 331: 141, 189, 1994.]

“ Are these results disappointing? Yes. Are they proof that antioxidants don’t work? No. But to some people, the failures meant that scientists were stupid, out to discredit vitamins, or both.

“ Puzzle Pieces

“ ‘True, the finding of harm from beta­carotene is puzzling, and may well be refuted by future trials,’ wrote New York Times science editor Nicholas Wade in an op-ed piece last May. ‘But that cannot be presumed.’

“ How can Wade be so unfazed when people are tearing their hair out trying to get some straight answers? Perhaps because he understands how research works.

“ ‘What medical journals publish is not received wisdom but rather working papers ...,’ write New England Journal of Medicine editors Jerome Kassirer and Marcia Angell. ‘Each study becomes a piece of a puzzle.... No matter how important the conclusions, they should usually be considered tentative until a body of evidence accumulates pointing in the same direction.’

“ Proving Cause and Effect

“ ‘Ignoring dozens of positive studies published in the last decade, the media has given enormous coverage to a single badly flawed Finnish study which supposedly showed negative results,’ charged Citizens for Health, a lobbying group supported by the supplement industry, last April.

“ But unlike the Finnish study, virtually none of those dozens of ‘positive’ studies were randomized, double-blind, placebo-controlled clinical intervention trials, which actually give people antioxidants (or whatever) and then wait to see what happens. Trials are less likely to suffer from pitfalls (see page 8 [i.e., left half of mock board-game, below]).

“ For example, most of those ‘positive’ studies simply ‘observed’ lower cancer rates in people who consumed antioxidant vitamins, usually from a diet rich in fruits and vegetables.

“ But something else in fruits and vegetables could prevent cancer. Or people who eat them may do other things to protect their health.

“ ‘All that these observational studies really tell us is that there is an association between vitamins and health benefits,’ notes Wade. ‘For a biologist, that’s the statement of the puzzle, not its solution.’

“ In fact, it’s never just observational studies — or just any one kind of study — that scientists need to prove cause and effect. ‘Cause and effect can only be based on the totality of the evidence, from all research methods — clinical, pathological, animal, experimental, epidemiological, and, when available, well-designed randomized controlled trials,’ says renowned epidemiologist Jeremiah Stamler of Northwestern University Medical School in Chicago.

“ Yet the media and the vitamin industry pick and choose the types of studies that make their case.

“ ‘Four studies have been done on antioxidants and colorectal cancer, with mixed results,’ said a trade association of vitamin-makers last July. ‘In the largest study, involving 35,000 women in Iowa, supplements of vitamin E were found to be strongly protective against invasive colon cancer.’

“ A popular health newsletter also miscast the Iowa study. ‘The largest and longest trial to date, lasting nearly five years, found no adverse effects from daily doses of more than 1,300 mg of vitamin E,’ it said.

“ Only one problem: The Iowa study wasn’t a trial. It observed a lower rate of colon cancer in women who were taking 1,300 mg of vitamin E a day. It couldn’t prove that vitamin E — and not something about people who chose to take E — was responsible.

“ That’s not to say that trials always speak the truth. They, too, can reach erroneous results.

“ Confused? Our ‘Non-Trivial Pursuits’ game on page 8 [i.e., left half of mock board-game, below] provides a snapshot of epidemiology — the type of research most often reported by the media.

“ Of course, researchers don’t always follow the step-by-step procedure shown in our game. For example, much of the evidence that saturated fat and cholesterol in foods cause heart disease came from animal studies, small clinical studies (measuring cholesterol levels in people fed high-fat diets, for example), and studies comparing populations eating vastly different diets.

“ In fact, the case-control and cohort studies described in ‘Non-Trivial Pursuits’ yielded little support for the link. And in the early 1970s, researchers decided that an intervention trial would not be feasible. Still, there is almost universal agreement that saturated fat causes heart disease.Closing quotation mark

facsimile of Oct. 1994 healthletter spread (with mock board game)

^ Non-Trivial Pursuits, a mock board-game about the “zany, madcap world of Epidemiology, where you, the ambitious scientist, attempt to unequivocally prove your theory of diet and disease.” Created in 1994 (copyright Center for Science in the Public Interest).

Click/tap here to view a larger digital facsimile (961KB file) of the original tabloid-size (11" x 17") spread, printed in the October 1994 issue of CSPI’s Nutrition Action Healthletter, vol. 21, no. 8, pp. 8–9.

SOURCE:  Bonnie Liebman, “Non-Trivial Pursuits: Playing the Research Game.” Nutrition Action Healthletter, October 1994, vol. 21, no. 8, pp. 1 and 7–9.