Evaluating Health Information

     Found It - What Next?

Chances are that the source of a news story about health topics is a research report in a scientific or professional journal. Those are a genre of their own, and can be hard to understand - but they can be deciphered, with some tools and techniques.

A Bit About How Research is Done

First, it helps to know a bit about how health research is done, and how it's published. These links can give you some background:

How to Skim a Scientific Research Paper

Most scientific research articles are organized in the same way - and they make the most sense if you don't read them from start to finish, but read them out of order, section by section.

Step 1. Start with the abstract (the author's summary of their paper).

Step 2. Then skip to the conclusions and discussion sections at the end to learn what the research study found.

Step 3. Next up  -  if the article still looks useful - go back to the beginning to read the Introduction. This should give you background context for this research study, why it matters and how it relates to other research studies.

Step 4. And only then, if you still think this article is still relevant to your question, go back to skim the description of the research project and its methods. Be sure to note the size of the study (just a few? or thousands?) and if it was a controlled study - see the nearby box for more details.

Image credits:  Kapi'olani Community College's Library & Learning Resources webpage on how to read a scholarly article.

What are Clinical Studies?

A scientific investigation becomes clinical research when it studies humans (instead of using theoretical models, in vitro studies of cells, or by experimenting on animals). 

Clinical research could look at has happened in the past (population studies or retrospective studies), or at what is happening now (epidemiology). Or it could observe the effect of one or more variables, either in one person or a few people (a case study) or a group of people who share many characteristics (a cohort study), or a much larger group of people matched for one a few shared characteristics.

Clinical trials are when the research questions involve treating people or examining some particular intervention. As you can imagine, there are a lot of controls on clinical research - especially in clinical trials - to avoid causing harm to the people in the process of study. A clinical trial is the next-to-last step in getting approval to bring out a new treatment or medical innovation, and only happens if there are significant reasons to think it might work, based on earlier studies that did not involve humans. (Participating in a clinical trial, if patient and healthcare team agree that it would benefit, and if patient gives consent after being informed of possible risks, is how to gain access to experimental treatment.)

Research Methods That Should Raise a Red Flag (when it comes to medicine)

While these can be perfectly valid methods for scientific research, these methods are usually not considered to be sufficiently scientific to recommend a course of treatment on a person:

In Vitro or animal studies: research that uses cells grown under controlled conditions, or laboratory animals, as a "stand in" for humans. These kinds of studies are done to decide whether an approach looks to be both promising AND safe, before attempting research involving humans. If the research has not progressed to humans in a clinical trial yet, it's still too experimental to be used as treatment. 

Studies with a small population or case studies: research based on only a statistically small number of subjects. A group of 150 can be quite large for some things, but it's still too small to represent a true sample of all people with XYZ condition. It is too easy for unexamined factors (like age or co-existing conditions or environment) to skew the research results.

Instead look for reports of a clinical trial or a much larger study (1,000s of research study participants) before seriously considering that information to make a medical decision.

Another red flag goes up for studies without any control group.

Research that does not include a separate group of  people - matched as closely as possible to the study group - who did not receive the experimental treatment. Without that, it's extremely difficult to show cause; the best it can do is suggest correlation, and even that can be questionable.

If there is a control group, it's called a "controlled study". Comparing the study group to the control group is one of the gold standards for modern scientific research.  If the individual people in the study don't know which group they're in, it's called a "blind" study; and if the researchers conducting the study don't know either, it's called a "double-blind" study.

Be Wary of Bias in Research Studies

Even if research is evidence-based, or peer-reviewed, or a double-blind controlled study, it may not be accurate (or accurate for everyone).

Scientific research has some built-in biases. It's rare to find published research on failures, on what doesn't work - which can mean that studies are based on only one side of the research question. Survival bias happens in medicine, too. (This is particularly a problem with pharmaceutical research. Once a drug has been approved for use, it can be difficult to find published research studies on problems, side effects, or drug interactions.)

The way people were recruited into a study might mean that they share some characteristics. (Nurses are one of the most-studied groups, because they're close to hand at research hospitals; likewise students.) But that can risk leaving other relevant groups of people out of the study. (For example, research on the correct size of drug doses is first done on adults, because children can't give consent.) A lot of earlier medical research is based on studies of white men, not considering whether women and/or people of color would have different experiences.

Too often that means the people left out of the original studies don't get proper care.

The current COVID pandemic is shining a spotlight on health care inequities in the U.S.. One attempt to address inadequate care  caused by research bias is to encourage more diverse participants in research studies, like the National Institute of Health's All of Us initiatives.

Bottom line: pay attention to who is included in clinical research studies -- and talk with your healthcare team if you have reservations about whether a treatment or procedure is best for you.