Skip to Main Content

Evaluating Health Information

How to read and evaluate medical information (even if you're not a scientist)

     Found It - What Next?

Chances are that the source of a news story about health topics is a research report in a scientific or professional journal. Those are a genre of their own, and can be hard to understand - but they can be deciphered, with some tools and techniques.

A Bit About How Research is Done

First, it helps to know a bit about how health research is done, and how it's published. These links can give you some background:

How to Skim a Scientific Research Paper

Most scientific research articles are organized in the same way - and they make the most sense if you don't read them from start to finish, but read them out of order, section by section.

Step 1. Start with the abstract (the author's summary of their paper).

Step 2. Then skip to the conclusions and discussion sections at the end to learn what the research study found.

Step 3. Next up  -  if the article still looks useful - go back to the beginning to read the Introduction. This should give you background context for this research study, why it matters and how it relates to other research studies.

Step 4. And only then, if you still think this article is still relevant to your question, go back to skim the description of the research project and its methods. Be sure to note the size of the study (just a few? or thousands?) and if it was a controlled study - see the nearby box for more details.

Image credits:  Kapi'olani Community College's Library & Learning Resources webpage on how to read a scholarly article.

What are Clinical Studies?

A scientific investigation becomes clinical research when it studies humans (instead of using theoretical models, in vitro studies of cells, or by experimenting on animals). 

Clinical research could look at has happened in the past (population studies or retrospective studies), or at what is happening now (epidemiology). Or it could observe the effect of one or more variables, either in one person or a few people (a case study) or a group of people who share many characteristics (a cohort study), or a much larger group of people matched for one a few shared characteristics.

Clinical trials are when the research questions involve treating people or examining some particular intervention. As you can imagine, there are a lot of controls on clinical research - especially in clinical trials - to avoid causing harm to the people in the process of study. A clinical trial is the next-to-last step in getting approval to bring out a new treatment or medical innovation, and only happens if there are significant reasons to think it might work, based on earlier studies that did not involve humans. (Participating in a clinical trial, if patient and healthcare team agree that it would benefit, and if patient gives consent after being informed of possible risks, is how to gain access to experimental treatment.)

Research Methods That Should Raise a Red Flag (when it comes to medicine)

While these can be perfectly valid methods for scientific research, these methods are usually not considered to be sufficiently scientific to recommend a course of treatment on a person:

In Vitro or animal studies: research that uses cells grown under controlled conditions, or laboratory animals, as a "stand in" for humans. These kinds of studies are done to decide whether an approach looks to be both promising AND safe, before attempting research involving humans. If the research has not progressed to humans in a clinical trial yet, it's still too experimental to be used as treatment. 

Studies with a small population or case studies: research based on only a statistically small number of subjects. A group of 150 can be quite large for some things, but it's still too small to represent a true sample of all people with XYZ condition. It is too easy for unexamined factors (like age or co-existing conditions or environment) to skew the research results.

Instead look for reports of a clinical trial or a much larger study (1,000s of research study participants) before seriously considering that information to make a medical decision.

Another red flag goes up for studies without any control group.

Research that does not include a separate group of  people - matched as closely as possible to the study group - who did not receive the experimental treatment. Without that, it's extremely difficult to show cause; the best it can do is suggest correlation, and even that can be questionable.

If there is a control group, it's called a "controlled study". Comparing the study group to the control group is one of the gold standards for modern scientific research.  If the individual people in the study don't know which group they're in, it's called a "blind" study; and if the researchers conducting the study don't know either, it's called a "double-blind" study.

Tip 1 - Look for Evidence - Based Medicine

Evidence-based medicine (EBM) is a favorite phrase in healthcare today - but what does it mean? The Centre for Evidence-Based Medicine defines it as: 

 "the conscientious, explicit, and judicious development and use of current best evidence in making decisions about the care of individual patients". 

In other words, medical treatment should be based on scientifically-proven research, as up to date as possible while still being thoroughly checked. EBM relies heavily on systematic reviews and meta-analyses to determine the best evidence.

Look for any of these phrases:

  • evidence based or evidence-based
  • systematic review
  • meta-analysis
  • best practices

Tip 2 - Peer Review is QA for science

Think of peer review as quality assurance for scientific research. It doesn't guarantee that the research conclusions are right, but reviewers check to make sure that the science was done correctly. That process takes time, though - months, sometimes even years.

steps of peer review

Tip 3 - Avoid Preprints if Possible

Preprints of scientific articles are exactly that - articles written in hopes of being published. They have not yet been accepted by a scholarly journal, or they have been accepted for publication but are still going through editorial and peer review. These are advance reports,  not yet analyzed by anyone except their authors (or possibly an editor or colleague).

These are a great place to look for new ideas, intriguing new research methods, or suggestions for further research.

With an unknown disease like COVID-19, it is tempting for journalists and other interested parties to explore websites that display preprints to look for the latest information. By itself, though, information in a preprint is not safe to base someone's medical treatment upon. This advice , written for journalists on how to use information in preprints responsibly, is good advice to anyone looking for cutting-edge medical research.

Be Wary of Bias in Research Studies

Even if research is evidence-based, or peer-reviewed, or a double-blind controlled study, it may not be accurate (or accurate for everyone).

Scientific research has some built-in biases. It's rare to find published research on failures, on what doesn't work - which can mean that studies are based on only one side of the research question. Survival bias happens in medicine, too. (This is particularly a problem with pharmaceutical research. Once a drug has been approved for use, it can be difficult to find published research studies on problems, side effects, or drug interactions.)

The way people were recruited into a study might mean that they share some characteristics. (Nurses are one of the most-studied groups, because they're close to hand at research hospitals; likewise students.) But that can risk leaving other relevant groups of people out of the study. (For example, research on the correct size of drug doses is first done on adults, because children can't give consent.) A lot of earlier medical research is based on studies of white men, not considering whether women and/or people of color would have different experiences.

Too often that means the people left out of the original studies don't get proper care.

The current COVID pandemic is shining a spotlight on health care inequities in the U.S.. One attempt to address inadequate care  caused by research bias is to encourage more diverse participants in research studies, like the National Institute of Health's All of Us initiatives.

Bottom line: pay attention to who is included in clinical research studies -- and talk with your healthcare team if you have reservations about whether a treatment or procedure is best for you.

The text on this page is copyright PlaneTree Health Library, licensed under Creative Commons CC BY-NC-SA 4.0. Linked contents are the responsibility of their creators or copyright holders.