Are Fake News Detectors Biased?
Researchers found bias in fake news detectors against large language models. The study reveals a significant bias in many existing fake news detectors.

Illustration: generated for Physis Systems
Short answer: Researchers found that fake news detectors are biased against texts generated by large language models. The study was conducted by Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. The paper is a preprint that has not been peer reviewed.
The primary source for this article is arXiv cs.AI's Fake News Detectors are Biased against Texts Generated by Large Language Models. It is a preprint, which means it has not yet been peer reviewed.
What is the bias in fake news detectors?
The study reveals a significant bias in many existing fake news detectors. The bias arises from distinct linguistic patterns inherent to LLM outputs.
How was the study conducted?
The study was conducted by Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. The paper was submitted on September 15, 2023, and last revised on November 13, 2026.
What was the mitigation strategy?
The researchers introduced a mitigation strategy using adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news.
What do we know, and what is still open?
What is confirmed: The study found that fake news detectors are biased against texts generated by large language models.
What is still open: The impact of this bias on the accuracy of fake news detectors is still uncertain.
Why does Physis Systems cover this?
Physis Systems follows research at the frontier, from space to fundamental science. For more stories like this one, visit the Physis Systems blog.
About Physis Systems
Physis Systems (Physis Defense Systems, Inc.) builds Inventi Matrix, an invention engineering platform available in the United States and Canada at inventimatrix.com, and Lumina AI Companion, a personal AI companion you can meet on the Lumina page.
Frequently asked questions
What is the source of the bias in fake news detectors?
The bias arises from distinct linguistic patterns inherent to LLM outputs.
What was the result of the mitigation strategy?
The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news.
What datasets were released by the researchers?
The researchers released two comprehensive datasets, GossipCop++ and PolitiFact++.


