This morning, Marshall Allen of ProPublica helped us make more sense of hospital ratings. When an organization or facility pronounces a “tops in…” or “ranked one of the best in…” stop and ask — how was the data gathered? What are the comparison parameters? Who did the survey? Is it based on anything beyond opinion? Can it be quantified? Or for that matter, even verified? Are they presenting absolute or relative data? Or using a little of each when it suits the promotional purposes?
Then, Ivan Oransky of Reuters Health News and RetractionWatch.com, and Gary Schwitzer, who runs HealthNewsReview.org walked us through how to really understand clinical studies.
Journalists need to look at basic criteria – whether it was a human or animal study, the size of the cohort, how the stats are presented (absolute vs. relative again), whether it was observational, or self-reported data, or whether the math actually adds up.
Also, make sure comparisons are apples to apples: for example, Ivan discussed a recent study citing a statistical association of coffee drinking with cancer. The study compared a group of hospitalized pancreatic cancer patients that happened to be coffee drinkers with a “control” group of non-coffee drinking patients hospitalized for non-cancerous digestive problems. Pancreatic cancer patients may be coffee drinkers, however, those with other disorders may not drink coffee because of their other health problems (like stomach irritation) or other reasons; so if the control group did not have cancer and happened to not drink coffee, is there really a link? It’s not really a valid comparison.
Of course, various reporting basics were also emphasized, such as getting comments from experts other than study authors, being wary about outlier claims, not writing about early phase trial drugs as if FDA approval was assured, and making sure to obtain a full copy of the study, not just basing information on the abstract.
Lastly, make sure you understand the scientific jargon so you can simplify it for readers. If you’re not sure, ask. Even if you think it’s a dumb question. Because if you don’t understand the information, chances are good that your readers won’t either.
Our job as health journalists is to make sense of clinical studies for our readers and help them to understand what it means in the context of their lives. The first step is to make sure we understand it ourselves.
Today, a hundred or more journalists learned how.