Media dangers

I was talking with my wife yesterday about how scientific information gets distributed.  For those of you that have not been exposed to the process, it goes something like this.  One or more scientists author an article and then submit it to a journal for review.  The article is (typically) reviewed by three "peers" - although some journals only require a single sponsor, and there are are non-journal scientific publications that have no review.  One of the hallmarks of these types of scientific publications, at least in the peer-reviewed literature, is that the conclusions reached are not speculative.  That is to say, the conclusions must be directly related to the data obtained and involve no speculation (although sometimes they do and reviewers let it go).  This point is important because it is precisely at this point in the process that the media can mess things up. 

Here's a hypothetical (but typical) example.  An article is published in a journal that reaches the conclusion that 60% of the people in the study lost weight when eating margarine instead of butter.  Now, there's a whole host of factors that are not addressed in that conclusion.  For example, how controlled were the rest of the people's diets?  Surely they didn't all eat the same thing in the same portions throughout the entire study.  What about exercise?  How many of them exercised?  What exercise?  How long?  Who snuck in a extra-large meat lover's pizza and horked the whole thing one night?  How different were the various participants' metabolisms?

You see, a simply stated conclusion like this hypothetical one has lots of caveats attached to it even without the philosophical examination that we've been going through.  But, some reporter, looking for something to report, sees this conclusion and then on your nightly news you are told that eating margarine will help you lose weight.  It's harder to generate interest in watching a news program if the teaser is that there is a report that some people lost some weight eating margarine instead of butter but it's not clear if that means margarine is better for you or not.  Some media outlets are better at being accurate than others, but it's something to watch out for.

The reason, of course, that all of this matters is that people end up making all kinds of decisions based on these kinds of simplifications.  Perhaps more importantly, these sorts of reports give the appearance of black-and-white scientific conclusions being generated, when, in fact, it is often the case that the conclusions are not so straightforward.  These sorts of reports trivialize the process as well as the conclusions.  They are not, of course, going to go away, and they represent a dilemma for the media.  After all, the alternative for the most part is simply to not report, and that does not serve the public, either.  So, in the end, all I can say about this topic is beware.  Take all science reporting through the media with one or more grains of salt.  My experience has been that the primary value of the media is to let you know that a report is out there.  My advice, though, is that before you act on the media report, find the scientific document and read it for yourself.  Even if you skip all the technical parts and simply read the conclusions, you will, at least, be cutting out the media middle-man.  Thank the media for letting you know the article was out there, but don't rely on their report to konw what the article really says.