How to spot unreliable health research

From medical treatment to diet and lifestyle choice: how to spot unreliable health research

Amy Nimegeer, University of Glasgow; Chris Patterson, University of Glasgow, and Shona Hilton, University of Glasgow

One day we’re told that coffee causes cancer, the next that it protects us from it. Does this sound familiar? We’re all bombarded by confusing and contradictory health information every day – supposedly based on scientific evidence. But most of us have a difficult time assessing the quality of this evidence, particularly if it’s online. Many people who use health information on a daily basis haven’t been trained to appraise research critically, and even those that have may struggle to maintain the skills over time.

We find this worrying, particularly given recent efforts to make research more open and accessible. To try to improve the situation, we have launched a free online tool called Understanding Health Research to help guide anybody who wants to understand a health research paper through the process of asking the right questions, so they can weigh up the evidence.

There are real benefits to health literacy, which is the ability to understand, assess and use health information. These skills have been linked to better health outcomes, better relationships with healthcare providers, and better decision making.

However, critically appraising research is not just “common sense”. And not knowing the right questions to ask means that anything that sounds “sciencey” can hold the same sway, regardless of its scientific merit. While many health and science journalists do great work filtering out flawed and poor quality evidence, unfortunately plenty of bad health reporting is out there, and it can cause real damage.

For example, it can result in skewed coverage of good quality research and can legitimise unjustified claims. One way this can happen is when journalists introduce “balance” to stories, presenting opposing views. If the vast majority of scientists support a piece of good quality research and one maverick opposes it, a quote from each camp incorrectly makes it look like scientists are divided on the issue – a phenomenon dubbed “false balance”.

Asking the right questions

Change is urgently needed. Some journals are working to bridge the accessibility gap with plain-language summaries of research, but what can be done on the other side of the gap to empower people to engage with research?

It’s great that so many academic journals are now available freely. But how do you read them? Protasov AN/Shutterstock

This is where our tool comes in. Aimed at non-specialists, it guides users through reading, interpreting and evaluating health research papers. The quality of different types of research must be assessed in different ways, so the tool helps users to identify and ask the right questions to interrogate each of the major types of health study.

The tool begins with questions about generic, but important, issues like peer review, funding sources and ethical approval, which are followed with more specific questions tailored to the type of research you are reading about. For example, if you are trying to understand a clinical trial of a new drug, the Understanding Health Research tool will ask “Did the trial include a control group?”, alongside an explanation of what exactly control groups are. After you answer the question, the tool explains what the ramifications of your answer might by for the reliability and usefulness of the research. At the end of the tool, a summary page lists all the relevant positive and negative feedback to help you come to your own conclusions.

The idea for Understanding Health Research was born while reviewing existing tools and holding workshops and interviews with people who use evidence professionally – we noticed there was a lack of critical appraisal tools aimed at a general audience. We decided to develop something that could help guide people through the process of understanding a research paper, demystifying important scientific concepts such as evidence hierarchies, scientific uncertainty and the difference between correlation and causation.

To make the tool as user-friendly as possible, we collaborated with other researchers, web developers and the charity Sense About Science. We also worked with the health care consultancy company Minervation, which ran user-testing with people who might find the tool useful – including members of the public, carers, health advocates, health professionals, information professionals, and health policymakers.

The tool has been tested throughout the development process with many different types of users to ensure that it strikes the right balance of accessibility and usefulness. Now that the tool is finally going live, we look forward to gathering more feedback, and continuing to evolve the tool to make it as useful as possible.

Improving health literacy is an enormous and complex task with no one-size-fits-all solution, and Understanding Health Research is a small piece of a much larger puzzle. Ultimately, helping people to decipher scientific articles is only half the battle; it is responsibility of researchers not only to describe their research clearly but also to put their minds and imaginations towards developing more ways for people to get involved with scientific evidence.

Amy Nimegeer, Research Associate at the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow; Chris Patterson, Research Assistant at the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, and Shona Hilton, Deputy Director of the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow

This article was originally published on The Conversation. Read the original article.