Physis Systems BlogLumina AI Companion and Inventi Matrix
AI Research

Do Language Models Change Over Time?

A study examined how biases of large language models change over time. The study was conducted over 13 months.

Illustration of a large, glowing language model sphere with a subtle clockwork pattern, surrounded by shifting bias patterns

Illustration: generated for Physis Systems

Short answer: The study examined how biases of large language models change over time. The same model displayed a significantly different bias pattern after one year. Providers change and retire models at their discretion.

The primary source for this article is arXiv cs.AI's Auditing a Moving Target: Evidence Anchoring and the Expiry of Fairness Audits of Provider-Served Language Models. It is a preprint, which means it has not yet been peer reviewed.

What is evidence anchoring?

The authors developed a theory of evidence anchoring.

How do biases change over time?

The study examined how biases of large language models change over time. The same model displayed a significantly different bias pattern after one year.

What are the recommendations?

The study recommends a version record for every model response and re-audits triggered by model changes.

What do we know, and what is still open?

What is confirmed: The study was conducted over 13 months with 248,400 API calls. The successors of retired models carried the same bias as their predecessors.

What is still open: The expiry of fairness audits of provider-served language models is still an open question.

Why does Physis Systems cover this?

Physis Systems follows research at the frontier, from space to fundamental science. For more stories like this one, visit the Physis Systems blog.

About Physis Systems

Physis Systems (Physis Defense Systems, Inc.) builds Inventi Matrix, an invention engineering platform available in the United States and Canada at inventimatrix.com, and Lumina AI Companion, a personal AI companion you can meet on the Lumina page.

Frequently asked questions

What is the purpose of the study?

The study examined how biases of large language models change over time.

How long did the study take?

The study was conducted over 13 months.

What is the theory of evidence anchoring?

The authors developed a theory of evidence anchoring.

Watch

Inventi Matrix, the film Watch on YouTube

Get the weekly frontier roundup

One email a week from Physis Systems with the research stories worth knowing, each linked to its primary source.

We will email you a link to confirm. We never share your address.