Physis Systems BlogLumina AI Companion and Inventi Matrix
AI Research

Are Fake News Detectors Biased?

Researchers found bias in fake news detectors against large language models. The study reveals a significant bias in many existing fake news detectors.

Illustration of a large language model generating text on a screen with abstract shapes, surrounded by smaller screens showing news articles with simple shapes

Illustration: generated for Physis Systems

Short answer: Researchers found that fake news detectors are biased against texts generated by large language models. The study was conducted by Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. The paper is a preprint that has not been peer reviewed.

The primary source for this article is arXiv cs.AI's Fake News Detectors are Biased against Texts Generated by Large Language Models. It is a preprint, which means it has not yet been peer reviewed.

What is the bias in fake news detectors?

The study reveals a significant bias in many existing fake news detectors. The bias arises from distinct linguistic patterns inherent to LLM outputs.

How was the study conducted?

The study was conducted by Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. The paper was submitted on September 15, 2023, and last revised on November 13, 2026.

What was the mitigation strategy?

The researchers introduced a mitigation strategy using adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news.

What do we know, and what is still open?

What is confirmed: The study found that fake news detectors are biased against texts generated by large language models.

What is still open: The impact of this bias on the accuracy of fake news detectors is still uncertain.

Why does Physis Systems cover this?

Physis Systems follows research at the frontier, from space to fundamental science. For more stories like this one, visit the Physis Systems blog.

About Physis Systems

Physis Systems (Physis Defense Systems, Inc.) builds Inventi Matrix, an invention engineering platform available in the United States and Canada at inventimatrix.com, and Lumina AI Companion, a personal AI companion you can meet on the Lumina page.

Frequently asked questions

What is the source of the bias in fake news detectors?

The bias arises from distinct linguistic patterns inherent to LLM outputs.

What was the result of the mitigation strategy?

The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news.

What datasets were released by the researchers?

The researchers released two comprehensive datasets, GossipCop++ and PolitiFact++.

Watch

Inventi Matrix, the film Watch on YouTube

Get the weekly frontier roundup

One email a week from Physis Systems with the research stories worth knowing, each linked to its primary source.

We will email you a link to confirm. We never share your address.