Just like how we evaluate the credibility of news and social media, we must also assess the quality and sources of scientific evidence and research. As news is a more reliable source than opinion, there’s also a hierarchy for scientific evidence and studies, often based on methodologies such as study design, sample size, and the length of the research.
Science, like all human enterprises, is not free of error, bias, and misconduct. Science literacy helps evaluate the strengths and limits of peer-reviewed research.
Scientific and medical studies with larger sample sizes or carried out over a longer time are often considered more reliable. Peer-reviewed or refereed scientific papers are subject to scrutiny by an editorial board of subject experts and evaluated for criteria, including significance, novelty and excellence, by experts before publication.
Systematic reviews – A synthesis of all relevant scientific studies on a particular topic
Meta-analysis – a systematic assessment of the conclusions of previous research
Randomized controlled trials – a study that measures the effectiveness of new medication, interventions or treatment
Cohort studies – a study that follows a group of people with common characteristics over time to explore health outcomes
Case-control studies – a study that examines if an exposure is connected with an outcome
Case Series/Reports – a descriptive study to illustrate novel or unusual medical features
Editorials / Expert Opinion – scientific commentary based on the views or opinions of the scientist or medical practitioner
When it comes to scientific research, a systematic review – a synthesis of all relevant studies about a topic – and meta-analysis – a systematic review of previous studies that makes conclusions about an area of research – offer the most reliable and high-quality evidence. Editorials and expert opinions have more bias and less reliability baked into them. Additionally, sometimes, the results of randomized controlled trials get misrepresented in news coverage, inflating the significance of findings based on small population samples.
This video teaches you to spot questionable science reporting by news organizations.
In this humorous video, comedian John Oliver explores how scientific studies often get misreported and overblown in news coverage. A warning: this video may not be appropriate for all audiences. It contains strong language and discusses mature themes.
Why are editorials or expert opinions at the bottom of the scientific credibility pyramid?
Answer: Scientific evidence has a hierarchy. At the top of the pyramid are systematic reviews, which synthesize all relevant scientific studies or evidence about a topic. Editorials and opinions are commentaries based on the scientist's or medical practitioner's views or opinions. They do not require empirical evidence. They can also be biased.